Okay, I was going to answer, but Nero already answered for me. Basically, everything he said is what I would say. The only thing I'd add is that we are all culpable for the patriarchy, some more than others.
I think there tends to be a misunderstanding of what privilege really means, so I'll address that:
Here's the thing about privilege: everyone has some. Nearly everyone lacks some. So all of us trans women who had a rough time growing up among boys, well, that just means we didn't have cis privilege or macho privilege or whatever. Everyone recognizes the gradations of privilege based on looks and femininity among women; well men have that too.
I had an awful childhood, but it's still as plain as day to me that I have experienced male privilege. In conversation, people have listened to me even when my ideas weren't as worthwhile, or when I was saying the same thing as a woman who was being ignored. I was encouraged to go into math and science in ways that my sisters weren't, despite two very supportive, mathematical parents and a very progressive school atmosphere. I haven't had to worry about studying late on campus and walking home alone. However much I hated how I looked, nobody ever told me I need to lose weight, take better care of my fingernails, wear more or less makeup, etc. Nobody ever leered at me in a threatening way when I went to the beach. I haven't had to worry when I went to parties that I'd get something slipped into my drink and wake up to a man raping me.
Being raised a boy with sisters shows you a lot about how boys and girls, men and women are treated differently in this society. Most of it is subtle. It's just in the air; it's as easy as breathing for us all to participate.
No, I didn't benefit as much as many men I've known. I've always been seen as something between effeminate and just plain peculiar. And the pre-transitional lack of cis privilege (which is quite different from the post-transitional experience) sure as hell sucks, in a way I don't expect any cis person can ever fully grasp. But that doesn't mean I wasn't afforded male privilege.