I went off all meds, including HRT, for 3 weeks prior to GRS. I had crankiness, hot flashes, and the return of some temporary masculine traits (like the man smell, ugh!!!), but nothing permanent or that set back my transition progress at all.
Since, as others said, E can suppress T to some degree too, you're really not even going to see as much of a resurgence as I did; it's going to take T time to rebound, and even when it does, it won't be back to your previous levels. You should be fine.