I suppose at least they're acknowledging your trans status and that it's something you're going through. Lately the people in my life have been trying to attribute all of my problems to anything but the trans, or just trying to make out it's some deliberate deficiency or misbehavior in me, or even just attention whoring on my part... none of which is true.
If they want to believe it's because of that, it's probably a waste of energy to try to correct them. I notice if you tell people a list of your problems, the trans thing comes out on top as the most drastic so they'll prioritize it mentally, and probably think it causes the rest or is the "root" of all your issues.
Personally I believe trans does have something to do with SA, as I have SA too and much of my social discomfort comes from feeling like a bear in a dress. Not literally - I don't wear dresses or look much like a bear ofc - but I do feel as if I don't belong with either men or women and I've no doubt that contributes to the problem, even if the SA does have multiple roots.