This is the completion of week three for me on HRT (Spiro and Estadiol), and I can tell you that my sex drive has decreased dramatically. Sex with my husband doesn't even seem appealing at this point, let alone the thought of masturbation.
Like Dena said, you need to be in the mood or, in my opinion, have some sort of emotional connection to the act rather than just "getting it on" for the heck of it.
So, don't be alarmed at all the changes you'll experience mentally, physically, and sexually as you progress in your HRT.