A friend of mine had a GG girlfriend who was asked that and she was rather offended and cried her eyes out while telling us about it...
We tried to console her and tell her how beautiful she was but she couldn't let go of it and moved away a short time later
I randomly ran into her at a cafe a few years later in another town... She looked much the same but now had rather large breasts
Anyway, I've been asked if I was a boy or a girl at various times over the past 40+ years, I thought it was a somewhat rude question to ask, but didn't let it bother me much
The only time it really bothered me was when a guy I was attracted to asked (I was afraid to admit finding some guys attractive at the time) Hoping he'd maybe pursue it further I said "Possibly, that's for me to know and for you to find out"
He just looked confused and walked off and I was left to feel flustered and disappointed
People are just curious and sometimes don't realize the question they are asking is rude... That much I can forgive... If they are being rude and obnoxious purposefully... Then there might be some trouble