You shouldn't act behind your doctor's back. Instead, have a clear, honest discussion with them about what you want and what your goals are. If you want the least amount of E necessary for well-being, then ask for that and see what they have to say. E is more than just for feminization, it affects mood, libido, bones, memory, blood circulation, skin ageing and texture, etc. Not enough E in the absence of T might have negative consequences on your body as a whole, mentally too.