This is an issue for all genders. Cisgender people tend to see us as a lot less "sexy" when they know our history, because they don't see us as "fully a man" or "fully a woman". Julia Serrano talked about that in her book Whipping Girl. She called it ungendering.