I'm not a fan of kids. I have no interest in having them, raising them, or being around them. I don't have the patience for them, and I don't find them endearing in the way that most people do. Most of my interests aren't things that are kid-friendly, and I'm not willing to give up things that I actually enjoy for something that I have no interest in. There are times where I can barely stand my dog when he gets a clingy, so kids are definitely not for me. I've been told I would be good dad material. But I don't care.
Society by and large is made up of people with kids or who have had kids at some point, so obviously they're going to be a little upset or standoffish towards people who dislike kids or are indifferent to them. People get upset over all kinds of things that are inconsequential - when it's something as personal as children, even more so.
You'd be surprised though. There are a lot of businesses here that don't allow children, and as much as it annoys parents, they tend to be well loved by everyone else. You don't have to worry about kids being poorly behaved or parents refusing to parent. It's nice.
Google "childfree". That's how a lot of people would describe themselves. There are whole forums online dedicated to it.