A the beginning of my transition, a few years back, after being told not to use women's toilets by my (male) boss, I decided to go downstairs the building to the only one "wheelchair" toilet available. That one is a "gender neutral" one, and I was not offending anyone by using it.
Until my FFS by Dr Ousterhout, I used these public toilets, so that neither men nor women were offended by an ambiguous (therefore suspect) being using "their" toilet. I always made sure that no handicapped person in a wheelchair would have to wait while I was quickly doing my business inside. That was about that.
After my FFS, I could at last live full time, it was nearly three years ago, and nobody ever thought twice about me. I've been using the women's restroom ever since, without any issue whatsoever.
So, I guess the "gender neutral" toilets thing is probably for people still appearing ambiguous while transitioning. Anyway, it's not something new.
I do concur, though, that privacy is quite important, for everybody. I have witnessed first hand (I won't elaborate, but it is easy to guess) predatory behavior on behalf of both men and women. Rape, harassment, this class of behavior is not specific to a given gender. But that's a different story altogether.