I find that it depends on my own mood at the time. If I'm feeling good to begin with, it generally doesn't bother me. Sometimes I simply look them in the eye and smile -- about half the time they smile back and we both move on. When they don't return the smile, I just tell myself, "That's a rude person," and I feel bad for him/her.
But what I learned a long time ago is that people look at people all the time! Most of us are "people-watchers" and so you and I should not assume that someone's staring means they're thinking anything negative.... I know that sometimes it's easier said than done, that "that" look can cut right through us if we're feeling less-than-confident; but honestly, I generally find that most folks just look out of passing curiosity or normal people-watching.
Of course if I'm feeling down, self-conscious to begin with, not at all confident in who I am, then EVERY look and glance scares the hell out of me. In those moments, I might lash out rudely to the starer -- but I don't like myself when I do that, because it means I've let the other person rule how I feel about myself. So, I work at feeling better about myself generally, which usually leads to not caring so much what others think.
As for children, they don't know they're being rude. They're just curious. And as Catherine Sarah said above, kids do see right through us, so try to not take it personally. Most often, I find that a smile aimed at them, even maybe a "knowing smile" as if we have a special secret just between us, works just fine.