People have a hard time understanding grey areas. I've heard the same thing about being bisexual - "you have to be one or the other, you can't like both" - and the same tends to be applied to all aspects of being LGBTQ - gay makes sense because you are a guy that likes guys, trans makes sense because you are a guy that wants to be a girl, but once you fall in between the binary, people don't get it.
Whether that "in between" is fluidity, androgenous, wanting to be both genders or neither, people will still try and push you into a box. I've run into it - "what are you, transgender or a ->-bleeped-<-?" Well, I want to have a generally female body, but I don't want to erase my male self, so can't I just be both? But the answer you'll generally hear is "no". I don't think that's right - to me gender and sexuality are a spectrum, not a series of binaries, but not everyone sees it that way.
Sent from my VS988 using Tapatalk