You're right to be cautious. We shouldn't (ahem) take these things lightly.
Conditions that require pharmaceutical intervention for the remainder of one's life involve serious thought . That said, and as Cindy mentioned, the typical medications we take have been proven safe - certainly much safer than the alternative (i.e. depression, etc.)
Grace also hits on a good point: if reducing your need for anti-androgens is your goal, then an orchiectomy is a cost-effective means to get to where you want to be. Opinions on whether this makes full GRS more difficult vary from surgeon to surgeon, although I do remember reading that if full GRS is something you're interested in later, there's a means by which to perform an orchiectomy that differs from the typical route and results in less tissue loss.
Taking meds for life is a scary thought, definitely. But compared to becoming addicted to alcohol, heroin, whatever helps you get through the day, taking E and AAs is way down at the low end of the risk scale if properly supervised.
