In your body certain cells have "Androgen Receptors" Poking out of their surface. like hair follicles, but also cells in your layrinx and penis and testisicles. Testosersone is a growth factor it comes in binds to those androgen receptors and encourages many different behaviours.
Like it might encourage hair follicle cells to grow and divide causing body and facial hair but on other cells like scalp hairs it's acts as an inhibitor to growth stopping cell cycle and encouraging apoptosis these cause on the large scale the changes accociated with the male puberty.
When you take a hormone blocker it ends up floating around your blood stream, it then sticks to this androgen receptor on those cells and stuffs it up breaking it the body will gradually replace the androgen receptors. without this "growth" signal certain cells can't live without it and undergo cell apoptosis. But some just stop growing so fast instead now reliant on other hormones such as EGF or other growth factors.
While some cells have androgen receptors others have "estrogen receptors" largely fat cells but some other cells too. That obscure ridge down the central line of your penis... Breast cells, Fat cells. When you expose those cells to estrogen they are encouraged to divide and grow. on the macroscale it manifests as breast development and the other changes we get.
The pituitary gland detects either estrogen or testosersone it dosen't care whitch you have circulating but will detect a normal level of "steroid" hormones and send shutdown signals to the gonads.