I had been freaking out about this myself a few weeks ago. I had just moved to a new town and for the first time I was on my own living in my own apartment. I kept visiting the local thrift store and wanting so badly to go browse the women's section of the store. Every time I went I would either make an excuse to buy something from the mens or from the non-clothing section. Three times I went, walked by the women's section, then pretended to look at men's shoes while stealing glances at the summer dresses. I was mortified that someone would see me and call me out, and I felt like a pervert who was doing something immoral.
Finally, I set aside two hours on a weekend and forced myself to go into the store and go right into the heart of the women's section. The first five minutes I was absolutely horrified, but I just focussed on the clothes and tried to hide my red face. The story was super crowded that day and I was sure I would be called out any minute by another shopper or by the intercom.
90 minutes later I hadn't been given so much as a passing glance by anyone. I looked through hundreds of dresses, skirts, jackets, and even a little bit of lingerie. I even picked out a skirt I liked, tried it on in the dressing rooms, and purchased it. The male cashier didn't bat an eye. I recently went into a Ross and purchased my first pair of panties as well, which was even more difficult but once again I was pleased to find that no one else cared but me.
It's tough the first time, but remember that what you are doing is not unusual, immoral, or illegal. You are the only one keeping yourself from it, and you shouldn't keep yourself from your own happiness.