I'll add a bit in here being from the UK.
I think the problem is that one of the threads of their argument does come from a position of feminism (or at least feminists) even if its one which most feminists would disregard. It would be very difficult to argue that Germaine Greer, for example, is not a feminist. She fits very squarely within second wave feminism and pretty much (quite literally) wrote the book on it. Many of the TERFs follow Greer's theories on feminism and, generally, reject third and fourth wave feminists, who in turn, generally, reject them back.
Its important to note that transfeminism very much was part of, or at least parallel to, the third wave of feminism. My personal view is that the viewpoints of Greer et al regarding trans women are, in fact, entirely anti-feminist (I consider myself a fourth wave feminist), and the vast majority of modern feminists would back up that viewpoint. Feminism, and what it means to be feminist, is not one thing, and there are different schools of thought and movements within it (although not a perfect similarity, its a bit like Wahhabism in Islam, leading to ISIS etc, but disregarded by all other threads of Islam teaching as being improper).
I wouldn't worry though about the framing of the argument. Whilst as trans-people we see these arguments all the time on twitter in particular, but elsewhere on social media too, in the real world, people don't know what a TERF is, people don't see this argument between the trans-community and this particular group of people, people don't think there is an argument between us and feminists. In the real world, most feminists are on our side, and these pathetic groups live in echo chambers on twitter or on websites like Mumsnet where they think that they have the upper hand but thats only because they repeat what each other says constantly. And they start ridiculous "arguments" like how they now suggest that they're #genderfree. Its pathetic and its seen as pathetic. This very small group of people in their little echo chamber do shout very loud in certain directions, but in the UK, where most of these groups seem to be for whatever reason, the public as a whole are on our side.
When I write, which is not as often as I'd like these days, I do use TERF. Not because I like the word, and certainly I'm not saying "I'm trans, they're a feminist, here is my argument" (as I say above, I'm very much a fourth wave feminist, and I distance myself not only from second wave feminism but also radical feminism) but merely because, from a descriptive point of view, people instantly know exactly which group I am talking about. To call them a TERF is not suggesting any validity in their belief as to feminism, any more than it would be to suggest that ISIS has a warped reading as to Islam.
My thoughts anyway
Em x