I was just recently in some shops and kind of wanted to look at the mens underwear. I was a little too embarrassed and was hoping my BF would look so I could just browse with him and if I saw something that looked like it would fit then I'd consider buying it. I was way too embarrassed to go look on my own or to even mention to him that I wanted to. So far although I usually pass as long as I don't speak to anyone as I wear gender neutral clothing such as jeans (always been mens of course), I've always worn women's underwear. So it just felt like an embarrassing step to say to my BF, despite that he knows I'm trans and is fully supportive.
Anyway, my point is I know where you're coming from, and I have to admit that the advice I'm about to give is something I myself need to heed!!!!

Last Christmas I bought my nephew some Superdry boxers. I didn't think anything of it. Women do buy mens clothes for other men. As presents, as partners etc, etc. So no one will bat an eyelid. (Need to keep telling myself this). Men also my women's clothes for their partners. Especially underwear, it's pretty common.
If you need help from the store assistant and you don't feel confident you could always say it's a present. Take your measurements in tell them your BF is this size and ask what should you get?
I think the problem arrises because YOU know it's for you. They don't.
Not sure any of that will help, but maybe it might a little.
Either way good luck (hopefully I might get some myself soon).
Oh and yes, online is always good and you can return if it doesn't fit. It might just cost you postage but that's worth it I think.