I wish I could say it's not important, but... Well, it's a survival trait. Given the current popularity of Trans-bashing, and the self-appointed Guardians of the Restroom (Yes, even in San Francisco. After the TransMarch ended, there were a couple of these jerks 'stationed' outside the women's restroom at a mall near the end of the March!), being able to move around in public without being readily clocked is a safety issue.
I hope things change. I really do. Meanwhile, I have to be really careful and aware of my surroundings. Fortunately old men and old women share a good bit in their facial characteristics. :/