I think people are just stuck in their ways. Previous generations, mostly, as well as current generations. They think it's wrong not just for religious reasons, but because they think that males should be men and females should be women. Anything else, they think, goes against social norms and are therefore wrong. Perhaps they feel a need to preserve the staus quo- that men should be tough and rough, that women should be gentle and delicate.
For instance, from Emily Post:"The attributes of a great lady may still be found in the rule of the four S's: Sincerity, Simplicity, Sympathy, and Serenity."
Manners are considered important in a "civilised" society, and those who do not go along with them are seen as undignified and rude. The same goes for genderbending: by refusing to conform to the behaviors of your perceived gender- such as a woman who burps, drinks beer, works with her hands, and never wears skirts- may be seen as rude and undignified. But across time and cultures, manners and gender roles chnge, which is why women are "allowed" to be much more assertive now than they used to, and men who prefer to stay at home to raise the kids may be seen as desirable.
