Well; a lot of the virgins I know, especially the ones in their mid-twenties, are simply so childish in their jokes. And sexist. And hopelessly disgusting in their remarks.
I think it's partly to do with yearning, and once they get over that, they'll see that people are just people, regardless of gender. That sex isn't this strange foreign thing, but a normal and loving act.
Also, it has been shown that having sex releases hormones that the brain needs to mature in certain areas. Having (had) sex often calms men down, for example, and solves a lot of anxiety issues. Interesting note here: It is hypothesized that only romantic sex releases these hormones. It has been shown that masturbation, for one, does not contribute to maturing psychologically.
To be honest: If you are a well-rounded non-judgemental patient and careful person, there is nothing wrong with being a virgin. If you are not, then losing your virginity might help. So yes; I think it is wrong to judge people on the basis of them being a virgin. People are people, and nobody is the same. But in some cases: They just need to get laid and get over it, especially when their behaviour is offending people around them.