We live in a binary world for the most part.
I don't think it is clear cut on the fairness thing though. Is it fair to compete against someone that has a clear physical advantage in their makeup - but then how is this different from one woman being physically better than another women anyway? No it is probably not fair on Caster, but perhaps it is not fair on the other athletes either - there does not seem to be a win win here.
Can we forsee a day where all female athletes in certain sports are intersexed in some way just to be competetive? That seems like a natural progression to me as there are advantages to having a testosterone fueled body when it comes to athletics (maybe not for things like gymnastics though). It is also interesting how people can run potentially faster on prosthetic legs these days. Maybe elite sport will end up being something only for the super super-duper humans, the modified humans, the enhanced humans, the intersexed humans. Maybe training will take a back seat to modification.. racing robots.