I'm 6 foot and easily read but I've lived in role for two years, have always used female restrooms (in the UK) and not yet had a single incident.
There, more than anywhere else, it's important to look confident and at ease.
It's a lot easier if you are accompanied by someone who is clearly a genetic female though and many times I have been.
If possible, when going in alone I watch the traffic for a few minutes and try and pick up on when the place is empty or near empty - the worst possible situation IMO is to walk in and find a queue 'cos they'll all have plenty of time to notice and study you
- The going in part is the difficult one - once you're in the stall you're safe and even if the place is crowded when you come out, you're on your way OUT then so who cares?
Part of the "problem" is that the ladies room is NOTHING like the Gents - girls loiter and chat to each other even if they're complete strangers.
There is one pub that I go into that has a beautifully fitted-out restroom - complete with a pair of large leather sofas!!!
I walked in there one night, having studied the traffic and concluded that there was unlikely to be anyone else in there, to find SEVEN girls sat on the sofas chatting

- they all looked up at me as I walked in so I smiled slightly and went into the stall. They carried on with their chatter and I did not hear any whispering or negative comment.
It's bound to happen that there will be an incident one day and I have a pre-planned outline of what to say, but...... so far, so good.
I can't stress enough though how vital it is to appear confident and at ease.