Hi Justarandomname,
I feel just like you, "when do you know your passing?", and does it matter? To me, it does, since I live in a more conservative part of L.A., the southern portion, blue collar part, but when I go to Hollywood, I have no problems, not giving a rats behind. Passing does matter, depending on what part of the U.S. or world you live in, at least for me.
I guess you know your passing, when you start overhearing people, address you more like her, she, instead of him, that guy, or the derogatory it, or worse.
Don't know if that answered the question, but I think I get your drift.