I used to think a lot along your lines, and it's a large part of the reason I used to be so depressed.
Over time however, I've come to have a more vague, but perhaps ironically more accurate sense of what gender means.
And that's really the question here, what IS gender? What makes someone male or female?
Is it genitalia? Because there are plenty of intersex conditions which blur that line. If you are born with both male and female reproductive organs, which are you? Are you whichever are the dominant/more fully functional organs? What if that question is a matter of debate? Genetically you're not fully one or the other from a genital point of view, but you will still tend to have an identity of one or the other.
Is it chromosomes? As another poster mentioned, there are conditions like AIS and Klinefelter's syndrome, If you have both two X AND a Y chromosome, which are you? Are you male because only males have a Y chromosome? Are you female because only females have two X chromosomes. Genetically you don't fit fully in either category. But before there was a knowledge of conditions like AIS and chromosomes, there'd be little doubt that a woman with AIS was female. If a woman with AIS looks like a woman, believes she is a woman, acts like a woman, in all ways that are apparent IS a woman. How could you say she's not? Because of her chromosomes? Is gender really that arbitrary? If you fixate on such a relatively minor detail to define gender, then what is the point of having a concept of gender at all.
Is gender social? That seems to ultimately be the only thing definitive. For all practical purposes, your social gender is the only one that truly counts in real life. Everything else has exceptions to the rules.
Your gender is not as arbitrary as your chromosomes, if it were, woman with AIS would have to be considered men, despite acting like woman, believing they ARE women, and for almost all practical purposes BEING woman to the point that a less advanced society would have nothing to base them being men on.
The point I'm getting at here, is chromosomes are just one aspect of the differences between men and woman, but it is neither the most important nor defining characteristic. To me at least, the defining and most important characteristic is whatever you present to be as in real life.
EDIT: One more thing though, if all the above isn't enough to convince you, there is a growing body of evidence that neurologically, and perhaps even genetically, MtF's and FtM's ARE intersexed to some extent, in a way that can be detected in the brain.
So is a MtF really a woman? Is an FtM really a man? In the ways that matter most? In my opinion, yes.
