If you've got somewhere to be and/or are with someone else, that should cut down on people following you. Incidentally, that's about the only way I'll be in a dress -- I'm kind of worried being "in public" in an environment where I wouldn't feel safe or accepted, especially by myself. At the same time, I don't tend to just hang out or wander around in public very much (nor do I have a strong desire to), so that helps.
For a while, if you're kind of nervous when you're dressing "cute" (especially if it isn't socially accepted, but even if it is, it can happen), then that can translate to more anxious (and potentially aggressive) behavior. It's likely something where you'll calm down after a while (not so much after a few minutes, but after many occasions) -- it is something I've experienced with clothes, as well as issues that aren't so confounded with gender... and I think it is something I believe is true for others.
In sort of a stereotypically-bachelor male fashion, one of the ways the clothes issue occasionally pops up is when I'm out of clean outer clothes, and I lament having to do laundry instead of "just" wearing a dress. It's not really that common of an issue, but it's amusing/disappointing when it does pop up.
I'm kind of weird like that.