Many people attack things they do not understand. If they think you are interested sexually, they may put up a defensive fence, thinking that will keep them safe.
It is really their problem, not ours. There are many others who have no problem letting us be ourselves.