This association came about because in the most general sense all of us are seen to be gender transgressors in some way. That, and the same people that beat, bash, and kill gays, beat bash and kill TG persons also. As the old saying goes The enemy of my enemy is my friend. On top of that, there is/was a large overlap of both common spaces and common ground. So that grouping in the political sense was about mutual interests and common causes, not being the same.
And sure, some LG persons are not real thrilled with us. By the way, you might have notice the opposite occurring in, say, this thread. But for the most part that's select individuals shaking their tiny fist in mock outrage on the internet, and they are not the people participating in public efforts.
If any of you would really get out in these situations you'd find that the worst is sort of a benign neglect, not any sort of outright hostility. And if you really hungout for long enough you might even find a lot of support. I know that when the TransMarch in SF turns up Castro that the bars empty out and there are a lot of gay men out applauding us and cheering us on - giving support and encouragement, and that's pretty real.
But from your basement it might not seem that way.