The first thing that comes to mind is that maybe you are looking or even staring at people and they think it's rude or something. I mean, the fact that you are noticing people looking at you tells me that you aren't distracted by your own thoughts. Your attention might be turned outwards and you might seem really tense and not relaxed, and that can attract attention and annoy people. Others can spot it if you are glancing at them, even out of the corner of your eye, and be creeped out by it. If that's the case, just try thinking of other stuff and get it off your mind. You can't go around trying to observe the way people perceive you and still act natural.