One thing that may have happened here is that you were just so apprehensive and nervous that you were just hyper aware of what was going on and maybe misread it?
The truth is in today's society, few people bat an eyelid if you shop in the men's section. Plenty of non-trans females shop in the men's section. Hell, even before I identified as trans I always shopped for men's jeans because women's would never fit my muscular thighs. I'd have to go up 4 sizes before they'd fit on my legs and still they were uncomfortable. So I was obviously female and shopping in the men's section. No one has ever cared. I've shopped in the men's section since I was a teenager. I mean going back it's obvious why but I used to come up with excuses in my head for it in terms of things like, the graphic tees are better in guys section. More geeky/video game/film stuff, and so on. But never has anyone given me an odd look.
If you have problems with going to the fitting room, just be confident and either don't care (because trust me, the workers for the most part won't give a crap) and if you're really nervous you could say something like "My brother told me to pick these up for him and he's about my size so I just want to see if it might fit him. Save me having to bring it back hopefully. He's a picky bugger!"

I've worked in stores myself. It's extremely boring. I'd chat to colleagues all the time and we'd pick up where we left off on some story or chat about some TV show/film we watched. There's a good chance whatever they were laughing and s->-bleeped-<-ing about had nothing to do with you but in your hyper aware state you've thought it was about you.
It happens all the time. As soon as something negative happens we start reading into everything around us and assuming that because that one person glanced our way, that they hated us, or could "tell" without thinking that they merely glanced at us because we were looking at them and acting funny.
For all you know the girls may have been s->-bleeped-<-ing thinking that the attention you were giving them by nervously looking over and glancing their way was because you fancied one of them! Lol.
Don't let something like that get you down. You have no idea what those girl's deal was, nor should you care. What difference does their opinion mean to you in day to day life? Even if you shop there again there's not even a definite chance of encountering them again. In the grand scheme of things, regardless of what actually went down, they're still insignificant in your life.