It may be something called KP
I have it, and have found ways to combat the stuff, do you have any on your upper arms?
upper legs and arms are the most common place for it
its more common in females, too!

its called keratin pillaris ? and is caused by the excess keratin forming in the follicle causing a hard plug in it
I never noticed it until I started shaving my whole body, I found that shaving aggravated it it as well as smelly lotions and most creams. Its like shaving the peaks off of little mountains and just gets worse. I tried several products and settled on 2 things that someone on the web suggested
I only wash with Dove Sensitive Skin Nourishing Body Wash (this stuff feels amazing) and I use Amlactin Lotion with the green label, this is a exfoliating lotion and has cleared my KP up in about a month. These 2 products have made my skin super soft and great feeling to the touch. I dont use anything else as much as I love smellies in lotion form, I must resist

the Amlactin smells kinda bad, so I use it first thing in the morning and let it soak in for a hour or so before showering. I used to put it on after, but I could smell it all day and its odd. Bottle says twice a day, which I did for the first 2 weeks, now Im once a day. Its expensive at 15 USD a bottle. Then every couple days I exfoliate with a loofah. Ive started amlactin on my face for the last week and my god, my face is so clear and smooth as well