Since I've never carried a gender identity (well, I schlepped one around for a while, but it didn't fit right), one of my biggest issues was simply not thinking like other people. In terms of social intelligence, the androgynous have the advantage of being able to look from outside the box, but have the disadvantage of not being in the box.
Also, knowing my body will never quite fit my soul. That's a bit of it right there. If I could do a Ranma 1/2 thing, that'd be cool - I mean, if I'm going to be tethered to something that just isn't me, I might as well be able to choose for the moment. A woman's body might not be any better than a man's body, but it certainly breaks the 29 years of tedium.