I think you're holding on to some ideas that I'm not sure you're even aware of. For example, there are countless accounts of women who would dress up as men, and play the entire male societal role, just so they could do some profession that they really wanted to do, but were denied since they were women. Women who wanted to be actors, or musicians, or even soldiers, or countless other things. They changed their gender presentation, and sex characteristics (visible anyway) to be male 24/7. Is that creepy? Why is it creepy for a man to do the exact same thing? Because we can't figure out the motivation? Easy to see why a woman would do it when they were oppressed. So I think that sometimes we rationalize that it's okay for people to do this gender-crossing stuff when we can understand their motivations, but it's not okay when we can't understand their motivations. But that has nothing to do with them and everything to with our pre-conceived notions of gender.
Let's say you were born as you were, woman in a male body. Except you were abandoned at birth and somehow managed to survive on your own in the wild. If we accept on face value that as a 'woman' trapped on a 'male' body, you would still suffer some form of gender conflict. But how would you know you were a 'woman'? You would just know that *something* about the way you feel doesn't match with the physical aspect of your body. That these things are 'woman' and 'male' is secondary, you find out what those mean when you see other people performing their gender roles in society. And you say, yes I see how that woman acts, and that's how I act, so even though I have male physical parts, I identify as a woman because internally I feel like she does. But even this has more hidden assumptions - namely that the women you encounter are feminine women, and not let's say, butch lesbians who drive cargo trucks. If those were the only women I ever met, I'm not sure I would say I wanted to be a 'woman' because that idea wouldn't really match with my internal idea. But I'd still want to be me, in all the non-male ways I see in myself right now.
That kind of paradox is why I can't really get behind this idea that I was born a 'woman' but with a male body. I was born me. I've never really felt comfortable with the stereotypical male role, or with the more male body parts (in my case I have some pretty feminine aspects aside from genitals and face area). The truth is that I love lots of really 'girly' things like dresses and makeup and fashion and the idea of motherhood/pregnancy, the list goes on. But I also like 'manly' things like computer science, violent video games, and sex with women (imagined only so far). But there are genetic women who like those 'manly' things, and genetic men who like those 'girly' things. So it's all kind of nutty. But transition feels right for me so far, so I'm going with it.