It sounds to me like you may have moved on from this particular social group. Some queer spaces can be downright hostile, particularly to AFAB people who transition to male. I've had people in these spaces tell me that I'm just a lesbian who's ashamed of being lesbian; a traitor who transitioned so I can oppress them as part of straight society with all its privileges. You should see their faces when I point out that I have always been exclusively attracted to men, and that by transitioning I have actually given up the straight privilege I never asked for so that society can now treat me like any other gay man. But my sexuality is besides the point, which is: pretty much none of us ever transition merely so that we can gain 'privilege'. You may gain some privileges by transitioning to male, but you definitely lose others by not being female, or by simply being trans. So it's not as black & white as they think it is.
People who throw accusations of 'male privilege' at us often do so as a knee-jerk reaction because they feel threatened by the perceived privileges they think we will enjoy over them, and they want to tear us down so that they can feel better about themselves. That isn't something that a friend does... it's something that an abuser does. I've found the best approach is to call their bluff, remind them of their cis privilege, and then write them out of my life.
I have passed consistently for about a year and a half now, having gone from being treated like an attractive cis woman; to being treated like a 'dyke'; to being subjected to a load of WTFery from people who can't tell whether I'm Arthur or Martha; to finally having everyone just chill out and ignore me like any other random guy (which is utter bliss). And because I pass so well, I never bother to out myself in my day-to-day life. I only ever out myself to potential sexual partners; nobody else has any right or need to know because there's nothing wrong with being trans.
So far, I haven't faced any major danger for having come out to someone. And if anyone accuses you of being a 'fraud' or 'dishonest', that is an ignorant, transphobic viewpoint so that person doesn't deserve your time or attention, except perhaps to educate them. When people meet me now, they meet me at my most authentic. The only 'fraud' I might be guilty of is the fraud of trying to live as a girl & later a woman... but that's not a fraud of my own making, because it's not like I had any say in the matter.
And please don't feel guilty about feeling sad about this: life is not the Pain Olympics, and everyone's pain is valid. If aspects of your life are making you sad, you're perfectly entitled to acknowledge that sadness and to have it respected. It's the only way you can work through the pain & come out the other side. You're perfectly justified in finding this hard. It
is hard. But in time, and with a new, more suitable circle of friends, you will come out the other side.
<Awkward manly hug>