https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1472878/For both sexes, the only significant risk is having insufficient sex steroids, be that Testosterone or Estradiol, and for prolonged duration. BOTH are important for maintaining bone mineral density. They actually work together to that effect, not independently.
However, per current studies, it is true that women are at greater risk than men for Osteoporosis. BUT, that could be somewhat skewed because hysto/oofo was real popular "just because" not so very long ago, and then it wasn't regarded as so critical to actually stick to HRT (my grandmother went without HRT for DECADES after and unsurprisingly has Osteoporosis). Then again, menopause comes much earlier than does significant andropause, typically... so par for the course, women are still more likely than men to be affected simply due to that factor.
In other words, you really shouldn't be worried about it. As long as you don't cease T after Hysto/Oofo for any great length of time you should be just fine. Even better off than those not ever going on TRT, really (T drops 1% annually after age 40; your levels, if you continue TRT, needn't regress anywhere near counter-effective levels, ever).
P.S. Your ovaries will NOT still be producing endogenous Estradiol while you're still on TRT. They atrophy (effectively shrivel up and cease functioning). What Estradiol you have in your system while on TRT will be coming from Aromatase conversion.