Ya know, what you describe sounds a lot like how I feel at times. I will say that I will never de-transition, and I am very happy as a female, yet I have increasing anxiety. You're not the only one with a few toys up in the attic dearest!

I used to get anxious sometimes, but after a few years full time I am anxious LOTS! I now realise I had panic attacks in the past, but they occur more often now. I even just had a long stretch where I hated leaving my house due to ridiculous anxiety, and I never showed up for a job I landed (actually I showed up but I cried in my car outside and made up a lie about why I couldn't go in).
I am in school trying to do normal stuff, but my self image is irrationally low because I know I pass just fine, better than ever, yet I hate how I look. My doctor speculates that it could be the hormones, and some research showed the prevalence of anxiety in women, so maybe that's the case. The fact is that as much as hormones change the outside, the changes inside are greater (brain chemistry seems especially sensitive).
Also I'm just paranoid about people knowing about me. The deeper I get, the fewer people know I'm trans. Now most people don't ever know I am different and I am soooo afraid I will give myself away. It is stupid; I KNOW I shouldn't care, and should be proud to be myself, but it's tough. I'm a work in progress for sure.
Its funny because my boyfriend (a ftm who truly understands) just tells me with a bloody sly grin on his face, "Hey welcome to being a girl. It
IS what you wanted..."