I'm not androgynous but I choose to ignore a lot of nasty stuff for the sake of peace and getting along. We've got enough practice with it that I can usually communicate mostly wordlessly with my daughter when I need her to just let people be stupid, and we talk about it afterward when we get to a safe space. Some people are just awful and we have to limit our interactions with them, but there are good people who come around slowly with enough info and subtle redirection. The hardest part is managing the influence of people who are casually vicious when I'm not around.
I'm scared a lot too. Sometimes bad stuff goes down, and hearing people calmly joke about it can be triggering.