You may just be paranoid. Just because a man stares doesn't mean they can tell your trans.
At the beginning of my transition, of course everyone could tell because I was just starting out.
So that stuck with me for a long time even after going full-time.
One day, I was out and about minding mine when a group of guys were pointing and smiling, a few, laughing.
I immediately thought they were making fun of me. I got the courage and went up to them and asked,
"what is so funny?" and they started giving me compliments and flirting with me.
Then I felt stupid and was embarrassed and started blushing, lol.
So, maybe guys stare at you because you're a hot bombshell.

And women, well, why would they stare at another woman?