First of all, based on looks alone IDK how people can call you a sir. At worst they may be suspicious but even at that point they should see you're dressed as a female and just assume you're female so that they don't offend you "if you are a girl" ... make sense? However, maybe your voice is killing it all and people just want to be rude.
Second of all, when people see you as female, the entire mood is different. They talk to you differently, especially men. I've noticed that men are much, much more intuitive than women on reading a trans girl. If people see you as female without a doubt they will ma'am you without hesitation. Of course there are those out there who are knowing of trans related issues and will maam you. This is when knowing and understanding how people (especially men) treat cis women comes in handy.
Third of all I have the same issues as you. Some days I think I pass and I'm the prettiest girl I'll run into all day. Other days I wanna hide under the covers because I'm fearful that today will be the first day I've been clocked in so many months. Honestly, I think part of it is hormones. Testosterone makes people confident... lack of it is gonna drain your confidence. Also, you have to remember that you've been looking at yourself in the mirror every day of your life for the past so many years. You aren't going to be able to help looking in it and seeing at least a residual effect of yourself... and for me it's so strong that sometimes I look in the mirror and ask myself what in the world changed... even though countless people have commented on how much I have indeed changed.
You can always get bold... go out and talk to people, then tell people you're trans and see their reactions. Or better yet, get a friend to out you behind your back and have them tell you the response. That is one way I know I pass at least 95% of the time, even on the days I feel like I look super crappy.