I can say that I felt this a lot, especially since there was so much time between my discovering that I was trans and the actual pursuit of that transition. I first realized it when I was 17, did all my research, knew it was what I wanted, but I held off on it for many reasons. I didn't think my grandparents (who raised me) would believe me since I was still a teenager. I thought they'd think it was just a phase, even though I'd unconsciously been dressing and acting like a boy since I was 6, even though every birthday cake wish and wish upon a star was "I wish I was a boy," and they knew that. Then it was the fact that I didn't have the money and I'm not sure there were any health insurances back then that covered transitioning (which I have now, thank God). Lastly, I was bogged down with the misconception that no therapist or doctor would approve me for this transition since I wasn't a lesbian (another thing now thankfully resolved).
Because of all that, I had to struggle - mostly privately - for eight solid years with this ever-increasing dysphoria. I would go through bouts of "okay, this is never gonna happen, so just try living as a woman" only for it to come back stronger than ever after a very short while. One of the worst instances was when I had to be the maid of honor at my best friend's wedding. My friend was going to let me wear a tux, but her now ex-husband wouldn't have it. So I had to sit there and get my picture taken while wearing a purple strapless gown, my burly factory-worker arms displayed in all their manly glory. I felt like a lumberjack in a pink tutu.
After that, I started wearing my binder more. At first, I was a little nervous about it, since I have a large chest and it's virtually impossible to completely bind myself. Then, at about a month before starting my hormone treatment, I found that I couldn't bear to step outside the house without it on, semi-ineffectual though it may be. I feel a lot better already since starting my hormone treatment, and I feel I've fully set into the mindset that I am a man, and that's extremely liberating.