Rafe,
I will tell you that it does matter, although like Melissa said it depends on how much, and where it is. I have been able to grow hair underneath my chin for a good while now even before T. But now after being on T it is thicker, not so sparse like what Dennis refers to. The rest of my face is stubbly now, but not dark enough to matter of show. So I decided to darken the hair with a eye liner pencil on the sides and above my lip, and it makes all the diference. It looks like a 5 o'clock shawdow. Before I did this I still would run into being called "she". Not now. Men don't have smooth faces, so that is what keeps people percieving we female, then ofcourse it ruins my day. I swear though, after top surgery, I will go out everywhere with my shirt open!
Marco