I can relate to this minus the children and the hrt. My spouse I don't think could deal with me being fully male to society. I'm lucky in that I'm pretty androgynous looking and I get to enjoy my masculine aspects and wear male clothing while still being myself and not actually transitioning. For me, it works. I deal with dysphoria in my own ways.
Hope you're able to find happiness and balance with your in-between state.