Last year I was kind of concerned about the amount of pills I was taking in a day, and since my testosterone levels were slightly below average to begin with, my doctor suggested that for a month or two, we try lowering my dosage of oestrogen slightly to see if we couldn't achieve the same feminising effects with fewer pills. Wasn't a good idea in the end, though, as it wasn't enough, and the little red spots that had covered my forehead since the onset of male puberty came back, my dysphoria began to really flare up again, and I got, among other things, morning erections again. >_< So by my own experience, yes, stopping HRT or reducing your dosage to one which is ineffective can cause a return, limited or otherwise, to being physically male. This was after only a year on HRT, though, and as others have identified, the longer you've been on it the less likely it is that you'll switch completely back to male mode. So your mileage may vary, as they say. ^^
