It's just a personal thing, I've always found dresses on most people to be weird. Sarongs I can cope with, it's western women's skirts and dresses that mostly disconcert me. But then I'm a trouser-wearing gal, so it's probably got something to do with me fending off everyone's expectations that as a woman, of course I should *want* to wear dresses.
The couple of times I've worn them, I found them physically very comfortable, just emotionally wrong.