I think if you went and surveyed a million people, trans and non-trans, gay and straight, from different cultures and upbringings and whatnot, each would have a different view of what a man or woman truly is. Do we know what makes a man or woman? Not necessarily. If you want to bring it down to just anatomy, sure, it's easy to see the distinction. But when there are men like Michael Jackson, and women like Janet Reno (no offense to any of their admirers, but...) I don't think it's really possible to say there's only one way to be male and one way to be female.
In my opinion, it's what you believe you are that makes the difference. Yeah, transgender individuals' upbringing might raise more questions than most, but if what we believe we are is male or female, despite what our physical appearances say to the contrary, that's who we are! If we're in between, we're in between. I don't recall anyone having said it was absolutely necessary to identify strictly as one or the other. I'm FtM, and am pretty darned masculine, despite my anatomy, and my brother, who is all male, is far more feminine than I (and straight, as far as I'm aware). Nobody conforms to the norm... if we all did, the world would be a really boring place.
Rafe