If a person is in any restroom to achieve sexual gratification of any type - they are in the wrong place totally. Objections and reporting to management/police is very appropriate, unless it's that type of establishment. If it is, why are you there and are you wrong in objecting?
If a person is in any restroom labeled closest to their gender identity/presentation for the purpose of bodily elimination, cleanliness, make-up adjustment and possibly girly gossip, they are in the right place. Anyone objecting, verbally confronting or physically assaulting any other person there is wrong and reporting to management/police is very appropriate.
In spite of authoritarian removal of restroom stall doors, personal privacy should be of the highest priority. Why you are in the restroom is your business, not mine.
Susan Kay