I've been using the men's room for quite some time now. Never got hassled, but I did avoid using public restrooms whenever possible. I always made sure to take a leak before I left home; and unless I really NEEDED to pee, I would nix the public restroom and wait till I got home again. I'm a lot more confident now, but it's been a gradual thing.
One trick that I used made me feel a little more confident: As long as there was nobody in the bathroom WHILE I was peeing sitting down, I would quietly raise the toilet seat AFTER I had finished. It was just a psychological thing that said to any newcomers, "See? I'm an ->-bleeped-<- like all the other guys!" Except it's much more accepted to leave the seat up in the men's room.
I have to admit that I'm a little worried about jury duty in a couple of months. I'll be trapped there all day, maybe for several days, and I won't be able to hold it till I get home! But I figure that I'll be post-surgery by then and much more confident generally. My voice should be even lower, my facial hair should be a bit more noticeable, and I won't need a binder anymore. I will have changed my gender markers by then, too. Plus, I might pack, just for the added confidence.
Anyway, I agree with everyone else that for guys it's safer to use the men's room. If you're a little too butch in the women's, the gals in there can make all sorts of trouble for you.