Hi, Chris. I am in a similar situation. I have a wife, six kids, an excellent career, an extended family that thinks very highly of me, church friends that would be horrified to learn that I am trans.
I was on HRT for a few months earlier this year. It was AMAZING! I felt human for the first time in my life. I stopped because of the same kind of fears you have. I was starting to develop breasts, and I got totally freaked out by the thought of nearly everyone I know rejecting me, damaging my kids' view of me, etc. Whether or not my concerns were truly valid, it scared me enough to stop.
For around 6 months I stopped HRT and tried to completely suppress and ignore any thoughts about being transgender or gender dysphoria. It was horrible, especially because I knew what I was missing on HRT.
Just last night my wife and I had a lengthy discussion about how I feel and we both agree that some form of HRT is needed. I will be contacting my endo about restarting HRT. However, I am still uncertain of the long term plan. In a perfect world, I would aggressively pursue full transition. However, with all of the complexity of my current life, I am afraid of the massive turmoil that it would bring.