It is just a custom because it is a custom. It is certainly a mandatory requirement in the military to remove your headgear indoors. The sole exception is when called to the C.O.'s office: you wear your hat entering his office so that you can salute him. You put it on in the hallway outside his office, and take it off once he has given permission to enter.
There are all kinds of rules about hats. Men were not allowed to wear hats in church, but women were required to.
Like most etiquette rules, it originated in Europe, and is still followed there more than it is in North America.
I was taught that only cowboys wear hats when eating indoors. It looks rude to me because that is the way I was raised, not for any practical reason. Recently, I went to a restaurant without a wig on, because I'd been outdoors all day and it was sunny and hot. I couldn't take my hat off without revealing my bald head, so I felt compelled to apologize to my colleagues for eating with it on. It actually prompted an interesting conversation.
