i have experienced havong and not having male anf female privilege, as well as binary privilege. it often takes me by surprise, as i don't realize that one groups has privilege in certain settings until i experience it in ways that seem extremely bizarre and meaningless to me.
there are some defaults in society which i don't always notice until they're practically thrown at me, because i don't treat people diffenrently based on anything other than personality traits that i have personally observed in that person. so it always surprises me when someone defaults based on someone's gender, appearance, skin color, clothing, accent, family background or other things that would not matter in an ideal world.
privilege definitely exists. but it is not something that makes people good or bad, and many who realize their privilege will even use it to help less lucky members of their society.