Either, uhm...
- Put it to female because no matter where you're at, your goal is to go within female numbers.
Or...
- Put it to female at the subjective moment when "you're mostly female". Basically, if you made a scientfic graph of your fat distribution, at the breaking point between when it's closer to female than male. That's not easy to tell, but I guess when your body (face and all the rest excluded) seems to be becoming more female than male and passing as such, then it's probably time to switch.
But honestly, I doubt it makes such a big difference, so your gut feeling is probably quite good enough. The only sure thing is that leaving it to male forever is not the right option (not that it matters hugely IMO), because contrary to bone, fat DOES change significantly on HRT.