Most of these problems I avoid by having long hair, wearing women's clothes, etc. But things like Kat mentioned (I feel like a suck-up today) are a concern, especially for a tiny little sissy like me. I probably can't even defend myself, let alone anyone with me. I also worry that people will expect me to lift heavy things.
What Chamillion mentions touches on a larger topic of women distrusting men more. It's usually justified, sadly, but it doesn't make it any more pleasant for me.