Using the toilet is a simple public right and necessity is should never be a question of passing, beauty, clothing or anything other than the identity of the user.
Those signs on the bathroom were never meant to be a legal barrier to anything. Relieving oneself is not an intimate act, it's not something to be embarrassed about or is washing your hands and checking your hair in the mirror. If someone for whatever reason feels uncomfortable with that, it is their issue, not yours.
Last night I went to a concert with my girlfriend and her kids and felt like dressing masculinish. Unisex T-shirt, no bra because it was warm, distressed jeans, plain black boots, no makeup and my hair was in a bun. Other than my apparent boobs under my shirt, i was looking pretty butch. I used the women's facilities twice, both times standing in line for a bit, with my girlfriend and with out, and i was never even glanced wrong at. I was talked to typically and smiled at.
If someone had voiced opposition to me being there, I would not have hesitated to set them straight.