They do not refuse hair removal to men. Many cisgendered men undergo this either because their skin is too sensitive to shave, they are tired of doing it or their job requires a hairless face more than what they can achieve with shaving (for example, someone with a very, very heavy and dark beard in the army sometimes gets scolded because they look like they have not shaved even when they did).
As for denying breast augmentation, it is possible that they do deny it. But here, anyone can have this and no letter is required whatsoever.
SRS, though, is another story. It implies the destruction/modification of healthy organs without a medical reason. One of physicians' primary guideline is "do no harm" and, without a medical reason, they should not do something that would harm the patient more than it will help them. Without any diagnosis (without a problem), however, medically speaking, SRS is removing the patient's fertility and, to some extent, removing some possibilities from them. Medicine is an exact science, and for a problem requiring a medical intervention that have some negative consequences on one's health, a diagnosis is required, end of story.
And until we know better, transsexualism is a condition that causes a generally high degree of discomfort and requires a medical intervention for its effects to be negated, partially or entirely. And no physical anomaly has been found. Therefore, unless another cause is identified, it is a mental disorder.
You can't remove the diagnosis from the DSM altogether. If you do, no invasive/organ-damaging/destructive surgery - namely SRS - will be performed, as in medicine's eyes, if there is no diagnosis, there is no (medical) problem, thus no treatment is required, and there is no justification for damaging the body. If you decide it's not a mental illness, you have to state exactly what it is, or else it's like you declare it's not a problem and individuals with it have absolutely no need for a physician whatsoever, like homosexuals, which is false.
The issue here is not whether it is a "mental" disorder, it's that it's an innate problem that cannot be classified as physical unless some breaking evidence is found, as every bodily system works well, and therefore falls in the mental disorder category unless it fits elsewhere.
Theorically, this does not apply to breast augmentation, as no definite harm is done to the body in the procedure and the body keeps all of its functions intact. The person is merely exposed to consented, limited risks, and this does not justify its being blocked. If it did, all purely cosmetic interventions would be blocked.
Apart from that, if no diagnosis justifies an intervention such as SRS, it will logically be regarded as facultative and purely cosmetic, just like botox injections. This poses serious insurance issues.
