Therapy might help. The thing is (and this is something I had to get cozy with), society is so ingrained with male and female (and nothing else in between). From the moment of birth a gender is assigned and then kids learn what is male and what is female from their parents, from other kids, etc. It would be nice if that evolves and people learn to accept that not all males may look "standard issue" and vice versa. In the meantime though, that still leaves the issue of people's perceptions. You can't control other people or what they think. You only have control over you.
There were points in my early life where my mom and I butted heads about me not being feminine, not liking stereotypical female things, etc. So I know how that is to. But I'm pretty stubborn when it comes right down to it so that probably played a part in leading me up to where I am now.
There are certain situations where I'll play along now. Obviously any time I have to deal with anything legal (drivers license, etc.) or anything medical I have to play along. But some medical people now I have told the truth to and they don't seem surprised or put off by it. The uniqueness of my own situation (which does include some physical stuff) kind of made it so that if I was truthful about it, it made it easier. Some professional/work situations I don't stop being myself but obviously they think I'm female. In other social situations, I'm much more comfortable about just being me and it doesn't bother me anymore that people might get some weird impression or something. For me, that was the goal I was shooting for though. Other people's misunderstandings I think would only effect me if I had to interact with them on a regular basis (like the situations you seem to be in) and knowing me, I probably would just make it clear to them that I'm not "typical" ... I don't like whatever it is they're doing/participating in ... thanks for thinking of including me, but no thanks.
Being stuck in a rural area is a bummer. But I can tell you that it's not like that everywhere. If you go to a larger city, there's a lot more potential to meet people who aren't so black and white, lots more activities, etc.