I have mistakenly passed all my life.
The first time I shaved was just a couple of days shy of my 18th birthday. Yes, it was an awful mass of terribleness and I'm glad there are virtually no pictures from those days. However, even while being staggeringly Amish, I still got ma'am'd face to face by people. The whole reason I was growing the beard out like that was to stop the ma'aming, as my macho denial. But it kept going because I'm short and had long hair.
When I spent last fall in and around Seattle, because of the different style of dress, while wearing male clothing and using my normal voice, and no makeup, no accessories, I was ma'am'd dozens upon dozens of times. I don't recall any sirs, probably because I don't want to

. But moments stand out to me. Waiting to get on a bus that just pulled up, a group of guys my age was there about to get on, and one of the guys said "Dude, let her on first." Or when I was at Wendy's at sea-tac airport and the woman at the register accidentally asked for the order of the elderly man who was behind me in line. He looks me square in the face and said something about being after this young lady. Almost every time I used the restroom and a guy came in after me, he'd see me and leave, then sheepishly come back in. One time three guys in a row did it while I was washing some grease off my hands. All in all, being up in WA was a delightful experience.

Heck, two weeks ago back here in Texas. I was buying beer at a convenience store and got the joke this girl was making at the expense of her boyfriend and laughed, "see! she gets it!" as she looks me in the eye and smiles. I just smiled and laughed more.
On the phone I think I've been 'sirred' 5 times in the last 10 years, and that's not an exaggeration.
I don't have a female voice at all. I have a rather soft male face, but it's a guy's face. I've made no steps towards transitioning other than shaving. But I've had long hair all my life, I'm 110 pounds before you count my metal implants, no muscles, and I'm about 5'4.
Some people really don't pay that much attention at all in every day life. I've actually wondered if these people don't think I'm one
damn ugly woman

I mean, anyone who actually looks at me should realize what I am biologically, but it's happened so often and for so long that my mother flies into a rage at whoever does it if she's present.
I don't really know what I'm trying to say, other than it seems like the most key factor for passing in my experience (not that I've ever actually
tried to pass) is to have a small frame because half the people out there don't seem to even look at a person's face when they interact with them. I have mixed feelings about this, both because the thought of "Haha, I passed accidentally!" is half the time replaced in my mind with "...I bet he thinks I'm ugly!" and because I feel like if I do begin transitioning, I'll develop a false sense of achievement at first because of how little attention most people pay, then some perceptive person will see right through it and it'll be an awful experience. The other thing I think about is that people ma'am me then have doubts and are too embarrassed to correct themselves because I don't correct them.
Meanderings.