The way I see it this comes down to empathy and responsibility more than anything else. This is a hard fact or simple truth that I feel anyone who is transitioning or out as trans needs to accept and embrace.
It doesn't matter who you are, what size you are, where you are, or how you present yourself, you're going to come across other people in society who just don't have enough empathy or a sense of social responsibility.
This is so not your problem.
This is not about passing. If you think it is let me get you a juice, sing you a lullaby and snuggle you up in your blanket of social privilege and illusions of social problems caused by the victims. Passing means conforming to whatever social standards are there influenced by cultural beliefs and buying into the same beliefs of social privilege.
I'm sorry but neither cisgendered men nor women are overly concerned by how well they pass to others, they just are. I don't see why it has to be any different just because someone is trans.
This is a social issue. It's evidence of just how much distance is left for society as a whole to go to embrace diversity.
It's hard when so many people willingly buy into all the profiling and stereotyping that goes on with the social programming and oppression from so many authority figures (government, religion and media). But the fact remains the same that nobody can get control of your mind and thinking unless you allow them to.
The stares, just like the pointing, the comments, the backhanded compliments and so on is just as much part of our reality as it is for people of a different ethnic background or skin colour, for the disabled, for the gays, and for anyone who is different and has the audacity (read courage) to stand out from the crowd just for being themselves.
Yes it's draining to constantly have to deal with it, to allow for it, and at times it can be upsetting and depressing. It can also be discouraging if you want to see it that way.
I don't. Nobody gets to control my mind but me. If people can cling to their illusions, assumptions and half-baked notions of who I am based on my appearance or lifestyle, then be sure I can do the same thing to them.
It's taken me some time and a fair amount of practice but I'm not bothered by the stares, the comments, the s->-bleeped-<-s, the pointing, and the prejudice.
In fact I'm cool with it, because it validates me just as much as the people who go out of their way to show their support. It's what makes me stronger and even more determined to continue being out as trans, to continue transitioning, and to continue living my life on my terms and not on terms imposed on me by the rest of society.
You see I'm out and transitioning for me, for my own personal reasons, so that I can function adequately out there in the real world and in society as the person I truly am. I didn't sign any agreement with anyone to be eye candy for other people. I didn't sign any agreement or agree to conform to any standard of femininity and no different from any woman out there I'm female by my own definition.
This is how it is. You either take it or leave it, I'm not bothered. I'm sorry if my being trans makes you feel uncomfortable or upsets you, but it's really not my problem and not my responsibility.
My suggestion is not to let it get to you, not to let it diminish you, or affect the quality of your life because if you do you're taking on the responsibility that in my opinion rests quite firmly on their shoulders, not your's.