That is a good question!
I did a lot of research and talked to many people about this. In talking to my parents, they were concerned that this may have had something to do with how I was raised. Was I shown enough love, praised for my achievements, hassles at school, etc. I have assured them that it has nothing to do with that.
The best explanation I have found came from a group at MIT who were studying genetic factors that influence fetal development in the womb. They did not have definitive answers but their research pointed in a certain direction. I am certain that the current administration has canceled that research since the President claimed the project was making rodents transgender.
This is what I learned: (pass the popcorn).
In the womb, from the time of conception, everyone is essentially female. They have a set of gonads, nipples, and breast tissue. The first thing that develops is the brain and central nervous system, so it develops along that line as well.
They don't know which gene it is, but the belief is that a certain gene decides how sensitive the brain is to testosterone. Genes work like switches. Flipped one way and you get green eyes, flipped the other way and you get blue eyes. (This is an analogy, not exactly how it works). So if this unknown gene is turned on, the brain will begin to develop along the lines that are typical in males. If the switch is off, the brain continues to develop along lines that are typical in females.
This only affects brain development. Keep in mind that hormones do not play a major role until puberty. Here we are talking about development in the womb. After about the ninth week of pregnancy, the gonads develop. If there is a Y chromosome they will descend and form into testes and begin producing testosterone, which in turn, causes secondary characteristics like facial hair. If there is no Y chromosome, the gonads will develop into ovaries and begin producing estrogen and progesterone. That is the only function of the Y chromosome, either testes or no testes.
In most humans, there are 46 chromosomes, 23 from the father and 23 from the mother. The mother always provides an X chromosome but the father can provide either an X or a Y. The unknown gene may have set the brain to develop along the female line while in the presence of a Y chromosome. This is rare, but it is believed to be the reason for the incongruence between brain and body development.
Most people have heard of Down's Syndrome. It is the most common genetic "anomaly" in humans. One out of every 700 births in the U.S. have Down's Syndrome. What causes it? An extra chromosome. So instead of having 46, they have 47 chromosomes and that causes many differences in the way the human body and brain develop. But what is different is how much the person is affected, from living a "normal" life to severe disability.
A similar situation happens with Klinefelter's Syndrome. They also have 47 chromosomes, which results in not just XX or XY, but XXY. Just like in Down's Syndrome, the person can live a happy and "normal" life, or this can have a profound impact on the body and brain.
This does not mean that being transgender is a genetic condition and they can start testing people for nefarious purposes. What it means is that there is an underlying genetic factor, as yet unknown, but that only sets up the possibility of them being transgender. Because not everyone who is transgender experiences gender dysphoria. Of those who do, they all experience it differently in how it is triggered and how severely it is felt. For some, it might be a minor annoyance, while others find it utterly intolerable.
Disclaimer: I am not a doctor or scientist. I just read studies and research papers to try to understand what I am going through. And like you, CJ, there are still a lot of unanswered questions. But this is the best explanation I have found so far.