I really think that the bottom line regarding reading sex/gender is threat assessment; our brains are probably hardwired in certain ways that make women more likely to clock trans men (I talk a bit about trans women below). We all have basic survival equipment, but (generally speaking) women need to be vigilant about certain things that men don't, and vice versa.
In terms of survival and competition, unless he's desperate for a mate, a man doesn't lose anything by reading someone as male. And anyway, he's more likely to find a good mate through social contacts, not rape.
But we're talking about casual encounters here. If you're female, you're not a physical threat to him; if you might be male, you might be a threat. Since, in the short run, survival must take precedence over procreation, he's actually better off reading you as male in a chance meeting because you're a bigger threat to his physical well-being if you're another male. Not to mention that another male is also a threat to the first guy's access to the pool of available females.
On the other hand, in a similar casual encounter, a woman could have everything to lose if she should have read someone as male but didn't.
Some might argue that if all of this is true, why are men so threatened by trans women? I think some of that is social conditioning/homophobia, but the same survival rules apply. If he read you as female, he thinks you are not a threat and that he can not only dominate but procreate with you. If you "tricked" him, it's a double whammy. Not only can he not impregnate you, but you were a real threat that he didn't read properly.
If it's true that women are more likely than men to clock trans men, then that would explain a lot of things. Has anyone done a study to find out whether it is true?
Trans women are a different situation altogether. In the long run, it's safer for both men and women to clock a masculine-looking woman as male. I wonder if anyone has studied this as well.
On a more humorous note, one could easily argue that if men are not good at clocking trans women, it's because they're thinking with their junk instead of their brains.
