The prenatal hormone theory is the one that makes the most sense to me from a scientific perspective, though biology isn't my area. I'm sure it's more complicated than that, but at a basic level I think it makes sense. Might be some other genetic factors involved too.
I compare it to hardware/software, since I'm an engineer. All the discussion, even in scientific circles, is always exclusively about the body's hardware. Well, brains have their own kind of software too. Some is instinct, some is involuntary lower functioning (what I call the body's firmware, stuff that governs heartbeat, etc.), and some is developed over a person's life. Naturally the physical state of the brain affects how this works, and science still has much to learn about brain hardware. I've never heard anything at all about brain software. Psychology is the closest we currently have, but it isn't quite the same.
So my theory is of course that some people are born with incompatible hardware. A person is not a body, but rather the consciousness, or soul if you believe in it, that exists in the body. Brains are just neural networks after all, so I think a computer is not a bad model for a brain. You would miss alot if you considered only a computer's hardware and none of it's software, and I believe that medical science does exactly this when it comes to people. Everyone is different though, and neural networks, either alive or artificial, do not work in the same way as a PC. You can't just install stuff on them, they have to learn it, so each person will in effect have their own programming language and OS. Thus, it is extremely difficult to compare one person's thought patterns with another's. Maybe that's why we can't read each other's minds.
And so, the point of my longer than intended post, is that the only way to really know a person's gender is to ask. You could guess based on physical attributes, but you could be wrong, and a person knows her/himself better than anyone else can.