Yes, unfortunately, I have felt that way before. Shortly after I came out to my wife, I was having really bad anxiety issues, I started to play out all of the horrible scenarios that could occur, in my mind. It was terrible, I considered just going back into hiding and try to suppress these feelings again. Even though I have a very supportive wife, it still scared me.
One thing that helps me cope with this, is I can do this at my own pace, that I do not need to rush into it, I think that is what scares me most. And as I do this, I can transition alongside of my family. Not to say I want to put off HRT, if I could do it now, I would be doing it. Finances just make it difficult, especially being close to Christmas.
I remember in the 90s being told that you had to live as a woman for a year, before you could even begin. That really scared me off from it all, it was outright a cruel thing to require from a person. A person should be able to progress at their own speed, rather than be thrown out there.
I think one of the things that really got me rattled, was my wife was really gun ho about it all, which I appreciate, and do not blame her at all. I'm the sort of person that when my mind is set on something, I like to fully immerse myself, but this is different, this is something very personal, that has years of repression involved.
I don't know if this helps at all, I hope it does, as it does for me. It is not a race, you do not have to face all of the challenges at once, take it one day at a time.
I really do wish you the best, I also wish I had more advice that would help you out.