About 3 years ago (i.e. before I became able to pass well), I went to a conference at a Unitarian church. As soon as I stepped inside, a nice middle-aged church lady walked up and said, intending to be kind and accommodating, "We have transgender restrooms downstairs." She had probably been waiting some time for a trans person to walk in, just so she could offer that, it would be her good deed.
I just looked at her and replied, "Thank you, ma'am. I use the ladies' room." She was caught off guard and stammered, "Oh... that will be fine" and quickly retreated.
She'd learned to be charitable toward trans people, but one concept she had yet to learn was they don't all want to be placed in a separate "transgender" category-- most of us just want to be known as our target gender. Well, it was my fault for not passing.