Estrogen WILL cause your breasts to develop, period.
AFAIK , little actual fibrous tissue is laid down during the first 6 months of Estrogen and if HRT is stopped in that time any fat laid down there under it's influence will get re-absorbed or redistributed under the renewed direction of testosterone.
- for Estrogen to be effective you NEED an anti-androgen as Testosterone blocks the effects of Estrogen.
Indeed, breast development will usually occur (slowly) if you take only a powerful anti-androgen without any extra estrogen as the small but significant amount of estrogen naturally in the male body is enough to do it - it is the testosterone that holds it in check.
Read this very interesting article by Dr Richard Curtis
http://www.gires.org.uk/assets/Medpro-Assets/Progesterone.pdf