An inkling here too, as far back as I can really remember. But not until the past couple of years has it really turned into a non-stop struggle to stay sane as a female inside a male body and a male life.
I don't know what causes so many people to realize this later than others. I'm in the same boat as you, as are so many others here - we "'figured it out" two, three, four decades too late. But late is the wrong word, because it's never really too late to be true to who you are. Perhaps it's just growing up; getting comfortable with looking at ourselves critically; learning to be honest with ourselves; not being scared of the answers anymore; being in an environment that allows for more freedom; maybe just sick of feeling so damn lousy and starting to look at why, or having exhausted all the other easy options; could be biological changes; or maybe just as simple as not being stuck in that busy turmoil of youth. Who knows why some people figure this stuff out earlier than people like you and me. For some, their feelings are far more pronounced and defined, but mine kinda loitered in the background for a long time and only tapped me on the shoulder a few years ago.
There never was a specific point at which I suddenly knew I was transgender, and no real point at which I suddenly said to myself, "Dude, you might be transgender." It was more of a journey of discovery (ongoing) with lots of examination of my life and experimentation with who I am and who I want to be, through which I'm just starting to feel more comfortable with who I am. And for me, that's the ultimate goal - to be happy (happier?) with who I am. And I hope you start to find yourself too, because although it's a journey full of uncomfortable truths, huge highs and lows, and wonderful revelations, it's a journey worth taking. (And the guys and girls on this site are amazing support too!)