Lack of sex drive is not a female characteristic at all. Females have sex drives, too, and there is absolutely nothing wrong with using what you've got! There is no residual change in t level or anything like that. While there may be a spike/trough of t level after orgasm, it's not enough to affect feminization.
The best metaphor I can think of to describe it (given the thread topic

) would be like a spring that you pull and release (LOL). The height of the spring representing T level.. The spring gets pulled up higher than it's start position, bounces down lower than it was to start with, and eventually levels out to it's original baseline. A.k.a. no change because the trough down after counteracts the spike up.
Personally I do this almost every day and I am not ashamed to say it. Actually in some ways after HRT I feel like my sex drive is better- or at least more interesting to me- because now with the lack of ejaculation I have become easily multiorgasmic. If you haven't tried it yet, I highly recommend!