For me, that hat has a LOT to do with my passing. As I started out with HRT, my passing wasn't that good, which resulted in lots of unwanted stares. That definitely made me feel like I didn't belong.
Even after I discovered I had passing privilege, it took time. Took me around three months to realize I had passing privilege on first glance, which gave me the courage to go shopping a lot. I must add, this is also a step I took, to force myself to "try the privilege" while shopping.
Some months of shopping and living the life later, I gradually discovered my privilege was slowly extending itself to passing 24/7, even on second glance or after 2 hour conversations. This was a major impetus for me to start 'living the life' a lot more, and together with that, the feeling of not belonging slowly diminished.
Sure, I'm pre-op and after having been on hormones for just over 13 months, there are still things to be done. I haven't yet mustered up the courage to go in bikini (abdominal fat), and the pre-op thing is a dead giveaway when not tucking. But since SRS is coming up in 8 months, I'm pretty sure that'll fade as well!