The problem is not how "normal" people behave, I have been using the woman washroom when out and no one seems to pay attention or care. Rule of thumb is to use the bathroom of the gender you are presenting as.
The problem comes from to gray areas... work and surroundings.
At work, someone who is uncomfortable with a trans person using their bathroom can cry sexual harrassment (which is absolutely ludicrous). In this litigious society, we often have more to fear from the lawyers than we do from real threats. As such, employers are now faced with a two potential lawsuits... discrimination (trans person) vs. harassment (GG/GM). Hard to say which side they will choose, but in the end the employer always loses. And why? Because of some close minded idiot who cant handle being in the same bathroom as a trans person.
The second more "awkward" problem comes from the environment. It would be very uncomfortable using facilities where there are small children (think chucky cheese, etc). The last thing you would want is some soccer mom crying foul because her 8 year old is in the same bathroom as you if you present as female. Its not worth the confrontation.
Sometimes its a case by case judgement I guess... YMMV.