Honestly I ask the same questions. At one time I actually thought I was weak or lazy or afraid or a number of other reasons why I didn't have the drive to want to put myself and everyone I know or who cares about me through transition. When my brief flirtation with hrt did not work out I was angry about it for a short while. I wanted to have some subtle changes that would push me further into "androgynous male", however my body had some ideas of its own in that department. The whole thing just led me back to an inner knowledge that I've known since I was around 12 years old: if I want to be healthy, I have to listen to my body. I've been into natural stuff since I was about 21 (except for a few years of depression, anxiety, and bad decisions). I feel stronger about living as naturally as I can (as in minimizing and/or eliminating chemicals, etc. from my life) than I ever did about transition. When I went against my gut instinct and tried hrt, my body reacted badly. I even have problems with things as minor as cold meds, so it really was no surprise. Still it was kind of a last resort to make some physical changes and I never planned on doing it for longer than about 6-8 months. And even if that had worked out, I had no intentions of even changing my name let alone doing anything else to transition.
I have my own personal aversion to surgery. I wouldn't call it a fear so much as I'd call it knowledge. When you know medical professionals and surgeons you get kind of the inside scoop. When a surgeon tells you that unless they were in an accident and surgery was the only way to save their life they would avoid surgery at all costs it changes the perspective a bit. I'm not a big risk taker. Even though the odds are that surgery like breast removal usually goes smoothly, I don't need to tempt fate. I wouldn't consider it mutiliation personally, and if there was a way I could just magically wake up without these blobs on my chest I would cry tears of joy. But it's my choice and I choose to not have surgery unless it was a life threatening situation.
Another thing that's completely personal is that I know I would not be satisfied with changing my body via surgery or additional hormones now. I'm generally okay with my overal appearance but I know the body is never going to match my brain. So since I'm not in a situation where I have to transition to keep living at all, I work on just being cool that I'm a different kind of guy. It really helps that I've made a life for myself that I'm pretty happy with and that I do have friends who I'm out to and accept me. My life consists of so mucho more than just my gender, that gets me through a lot.