Just go for it! Prior to transitioning, I wore a 46 coat and 32 pants. I could do 20 pull-ups and 7 one arm pushups. After 2+ years of HRT, I can still do 2-3 pull-ups and at least 1 one-arm pushups. But my skin is soft, and the muscles are more rounded. A couple of days ago, I was in the locker room of a yoga studio where I am pretty much stealth (except for a couple of teachers and students, and I NEVER talk trans issues with them). This woman looked at my arms and said, "I have arm envy." I said I used to lift weights a lot. Then she started talking about having a personal trainer. You'll be amazed how much people just generally talk about themselves and really don't inquire a whole lot about you when it's just a casual meeting. (I live in So Cal, and it may be different in the mid-west or Texas).
I must advise, do fix how you walk. A masculine walk, with shoulders swinging, will make your shoulders pop out. You don't need to swing your hips wildly, just have a nice, feminine gait. Also, I can't stress this enough, fix your voice. I had a deep booming voice, but with hours and hours of practice and help from a speech therapist, I know have a soft, high pitched feminine voice. Everybody expects to hear a deep voice from a transwoman, and when they hear a realistic feminine voice, they see you as a woman.
Thats my advice for what it is worth. Go out there and keep your head up and life a beautiful life as an athletic woman.
