Twenty-three!
No, probably not.
Has to be at least four, though. For example, I consider myself an androgyne, both male and female. And as someone pointed out on here, that's like mixing blue and yellow: what you get is green, which is made from both blue and yellow, but by that very fact is neither blue nor yellow, but something else.
Then you could have someone who considers themself neither.
I think it goes beyond that, too. Are "boy" and "man" different genders? "Girl" and "woman"? I think so...if they don't have different pronouns, they nonetheless have different expectations, different roles, and so forth -- everything that should define a difference in gender. (In the interest of fair attribution, this isn't my idea. I got it from one of Kate Bornstein's books.)