Growing up speaking English in the USA with a Southern idiom, I was implicitly taught that Sir and Ma'am were the only appropriate addresses for children to use with adults and among adults not on familiar terms.
As I became familiar with other European languages, I began to see parallels with usages such as Sie/Ihnen in German and Usted/Lei/Vous instead of "du" or "tu."
Nowadays, particularly as more people specifically prefer to be recognized as neither gender, it's a little ticklish. I know what I want to be, and I don't want anyone to not want to recognize me as such, but neither do I want to guess wrong myself and hurt anyone's feelings.