It can be annoying for us, but bear in mind that a small child rarely questions out of animosity. (Older kids, a bit different as they have picked up on things from parent and playground.) The human brain includes some 'hardwired' image classification networks in visual processing, basic face classification included. I know that I can trigger conflicting signals in this primitive classification network, and the child can't quite tell if I am 'safe/female' or 'potential hostile/male'. (To oversimplify two possible responses from this network.)
I had thought about this and was ready for being questioned by a child. I've been approached twice by small children who were questioning. I answer in a soft voice that "I'm a human being, just like you and your parent." I get a little smile in response. They've gotten an answer, and I had a pleasant interaction with them, and perhaps it will affect their behavior in the future.