Erik, you make some really nice points and I definitely empathise with some of the feelings you mentioned.
When I was still kinda andro I was lots more self-conscious. The occasional negative comment could create a hugely disproportionate response inside. Someone might call me "Sir" and I'd fume about it sometimes for days rather than learning to do better. Eventually, as I learned to be less sensitive, those people became like a bad smell I'd become accustomed to. The trick here is to realise how it doesn't matter, which sounds obvious, but is also easier to say than do.
I guess the simplest way to explain this is that NO situation turned out as bad as I'd portrayed it in my mind. MOST turned out to be heaps of nothing! It was all speculation, and I'd psyched myself out time and again. The more I toed that line the more it wasn't bravery pushing me, but the realisation that things simply weren't as bad as anticipated. I figured the next thing wouldn't be so bad either!
Also, this is important: move and act with confidence, with a RIGHT to be there, and you'll see that people tend to NOT notice you. People seem to especially notice others who are trying not to be seen. Does that make sense? People don't take heed to someone acting naturally.
I've become more confident overall as a result of my experiences. I used to be quiet and introverted, but the path I trudged through myself taught me lessons that apply to pretty much everything in life. While I do still love my alone time, there are people who say transitioning won't change who they are, or who tell others, "I'm the same person I was then," well in my experience I am a completely different person today, and really better for it all.