I live in a subdivision on a fairly large property, with some space between neighbors. By and large (why do people say that?), my neighbors only notice that Marcy has a woman living with her, and don't ask about Mike. When I'm doing yard work, and it looks like an old neighbor is watching too closely, trying to make a decision, I throw a twig or something, perfectly girly-style, then they turn away. Girl face, earrings, women's tops and skirt, boobs, butt, etc equals girl. They just mind their business.
If I felt we were certainly physically in danger, we would move, of course.
Bev