For me, the fear of passing is because I'm afraid that people will correctly identify me as male, but then take a closer look and change their mind and "correct" themselves. But I pass almost 75% of the time, depending on the context of the interaction. I pass significantly less when I have to use my birth name (since my name change is taking it's sweet ass time).
Ever since I've had choice in clothing I've dressed as a boy. I had short hair since I was in elementary school. I passed as a boy as a kid, but I would say that I'm female just because that was what was accepted in that situation. There was a lapse in middle school (age 12-14) where I tried to fake being female, but I still never really pulled it off. I quickly learned that I was very much baffled by and displeased with those dumb adolescent expectations of females. That's also when I realized I liked girls, and through most of adolescence just assumed that I was a lesbian. I hated that term, and never felt it suited me, but it was the closest thing I could come up with for me since I wasn't really introduced to the concept of being transgender until late adolescence. It took a while for me to have the idea occur to me that that suited me. I think that was mainly because of the fear of what I would have to go through to transition (socially, mostly). Another major obstacle was the cost of transitioning. For a while there, SRS wasn't covered by provincial health insurance (in Canada). These seemingly superficial reasons were my main deterrents in transitioning before I realized how much of a toll being in denial was taking on me. Now I am 25, and after a LOT of thinking last summer, I started coming out to people. The past year has been a very eventful year in terms of self exploration and who the major people in my life should be. I don't imagine that will change much in the next while either.
I actively started trying to pass since last winter. I basically started after I learned about the criteria here to be eligible for government funded SRS. You have to live in your "preferred" gender role for at least a year before you can get surgery. I have always assumed that people view me as masculine anyway, so I didn't think it would be such a leap to start trying to pass. As much as I wish I didn't have to try to pass with my chest tumours, I knew I had to do it. How much easier it would be if I was flat chested like I should be. It was strange for me, though, because when I told myself I would try and pass, the rate of passing went up. Perhaps something to do with the fact I was acting more confident? But now I'm a month and a half on T, so hopefully I can get to 100% passing within the next year. I think my voice is one of the main problems I have. That and my chest, which is the bane of my existence. Binding only helps slightly, but that will be better in the winter I hope, since I will have the aid of heavy clothing. Plus I won't be sweating my balls off due to binding (as well as being fat). The dysphoria is very bad in the summer because I always think about how I have to wear two layers of clothing, and can't just walk around in a thin shirt like most men can.