I think what you feel is pretty common for many trans people (both ftm and mtf).
Long time ago I felt way uncomfortable too about remarks that questioned somehow my femininity. Knowing that was pointless (and unreasonable) to feel that way didn't help me either. Neither it helped being conscious that those triggering remarks were just casual/trivial comments, not aimed by any means to make me feel the way I felt.
It was I just haven't accepted completely myself. Nor I valued myself rightly. In that moment, it felt like any masculine trait (physical, behavioural or psychological) may threaten who I was, making me less worth both as a girl and as a human being. I guess that was because I assumed deep down that anything that was out of my rigid concept of "perfect" made me less worthy.
It's curious how ridiculous it can get. Not just because those things we perceive as "failures" (for not being completely gender-conforming) usually don't have any weight in any evaluation we could get. But because many of these "gender atypical traits" (at least that's how we perceive them) are regarded as being quite desirable for many (if not most) people.
You can bet I like masculine men; rugged, rough looking guys. Got a thing for muscle bears. I like body hair and beards (well groomed ofc), bulky frames, and a long number of masculine traits. However there are things (usually regarded as masculine) that annoy me.
For example, less sensitivity for emotions and less empathy may come in handy in certain situations, but you can trust me (bazillion girls support this) IT REALLY BECOMES UPSETTING when trying to figure out (or work out) things with you guys, not just in a relationship, but overall. Another thing (lol I guess this may become a bit of a rant about men so I promise to be brief and if not just excuse me) that often turns out to be maddening is what you guys usually pray as a benefit from T: more ability to focus, more sharp reasoning, etc. But when you actually spend lots and lots of time withdrawn and hyper-focused on something (job, hobbies) it may make us feel irrelevant to you (does it sound you guys? "I feel like you don't care about me", "I feel that I'm not important to you", long etc).
And not to say how dumb you can get to look (and how many times we girls just roll our eyes lol) when your -oh great- manly "hyperfocused sharp (and also linear as hell) reasoning" prevents you from getting the whole picture right or from seeing rather simple details. Even when Im able to admit that those annoying "boys things" may have certain advantages... I honestly would like for my boyfriend to have some more "feminine" traits, such as a big amount of emotional intelligence, a nice degree of sensitivity or a tendency to give things a little thought before acting. And a ton of girls may agree with me.
In spite of everything, first you have to accept (deep down) yourself for who you are (transness included I guess). Then it may become water clear that any of those things can't threaten you nor your masculinity. They won't make feel you less valuable. Once reached this point, you would be so ridiculously confident that the last thing you're gonna think is about nobody outing/clocking you. Just think about how you wouldn't give a crap on what people thinks about many other aspects of you that don't are related to gender/transness.
Just my guess =P
And for what is worth, I can understand jealousy towards cis people, I'm sometimes jealous (though in a healthy way I think) towards cisgirls too. But, in your case... honestly 90% of cisboys should (and you can bet they already do) envy you.