The problem with feminism is the same problem with most civil rights movements. People become too obsessed with attaching an arbitrary label to certain thoughts (be it their own or someone else's) and lose sight of the actual logic of the ideals associated with them. What does it matter if an idea is considered "feminist" or not? If it's a good idea, then it's a good idea. If not then it doesn't matter anyway.
The problem is that people feel the need to label and categorize thoughts for context. If they come from this group, it's automatically good/bad (general considered bad in these circumstances). They lose sight of what's being said in favor of who's saying it. Hence, the most universal problem with politics.