You are VERY much onto something here, and I agree with you totally!
We are NOT pathological. Ours is, in fact, a medical condition.
Very briefly, my story: Born in late 1955. Adopted at age 8 months. About 8 years ago, wrote to the agency which adopted me out, filled out their casework forms and got a synopsis of my birth history from the social worker assigned to my case. My birth mother was under immense stress at both my conception and delivery. I'm the survivor of an identical twin gestation and birth. Lastly, I learned much about my genetic-lineage medical history. It's good, fortunately. Also learned personality characteristics of birth parents. Interesting! Many things FINALLY made sense. Birth mom was petite, pretty, brilliant and about age 32 when my sorry butt was born. I am the product of an extramarital affair betwixt people in an on-the-job liaison. In the mid-1950s, this was a huge disgrace. Hence, the adoption. Get rid of the problem. Don't ask. Don't tell. Good bye.
Absolutely agree with you that we should be taken out of the diagnostic statistical manual. Also, I've worked in the medical records business for about 15 years now. I don't handle medical records. I create them. People like me are the nameless, faceless nobodies you never see and never know about who transcribe doctors' dictations when you stay in the hospital. I did hard time on the front lines of medical transcription for 11+ years. As of late, I've done quality control for medical transcription --- the best of the best of the best. We QCs are the LAST eyes to see your hospital report ... before it goes to the doctor for a signature. It HAS to be right by then.
Here's the point:
Because I work on the frontlines of medicine (I operate behind the scenes.)(Your doctor, your nurses and us transcription people are THE most important people in your life whenever you are in the hospital.), my opinion is an informed one on this particular topic. Over the years, I've done reports on many brain surgeries, neurological surgeries, research reports on the same matters. Neuropharmacological researches are now definitely establishing that our condition is most assuredly both a structural and biochemical anomaly. In plain English: There's a damned good biological reason why we are the way we are. We are NOT whackos.The researchers' findings are still being vetted out in the peer review process. That is why you haven't heard of them yet. It may be years before you do. In fact, it may be many years. Why? The Powers that Be don't like people like us.
Footnote: Researches find that sexual orientation is likewise biochemically determined.

Astonishing! We're NOT nuts! Like, no kidding?!? About time they figured that out!