I have similar problems: my anxiety levels are always sky-high thanks to the wonderful gift of Asperger's.

Also, I've spent my entire life worrying about how other people see me. I was constantly on edge because everyone expected me to act like a girl. Whilst I could put on a relatively good act, I was always terrified they'd catch me out (they invariably would) and I'd seem 'weird' to them. Now that I've come out and am doing my RLE I feel a lot less anxious in general because I don't have to fake being female any more.
I echo what others are saying here about talking to a professional about your anxiety. My experience with the NHS has been that they'll try to send you for CBT, which you may (or may not) find helpful. Personally, I've done it twice and it was a complete waste of my time because CBT requires you to examine your thoughts when you're feeling anxious... but because I'm so darned special, I only tend to notice that I'm anxious when I'm heading towards autistic shutdown or meltdown. By that stage, I can't think rationally so CBT is pretty ineffective for me. YMMV.
They also sent me for talking therapy; a maximum of 14 sessions with a trained counsellor. I spent most of that time telling him about my childhood trauma, my failed marriage and my trans feelings. I needed much more time to get the the bottom of my issues. So I left those sessions having barely scratched the surface of what I needed to resolve, and of course the NHS is pretty awful with mental health so I haven't had any further help since then.
Another approach your doctor may recommend is anti-anxiety medication. This can help too; it helps you cope by making you feel less 'bothered' about the various things happening in your life. I took them for a while and I enjoyed feeling a lot more numb than I usually do; it was a welcome relief from feeling so keyed up all the time. Sadly, I can't take these meds any more (they were affecting my photoepilepsy) so I now have to occasionally self-medicate with Chardonnay.

I'd recommend starting with your GP and asking what options are available in your area. Your GP is the gateway to all the services that could help you.