You could always use a jock strap: they're designed for supporting and keeping things in place (i.e. basically exactly what you're looking for). You can order them online as well and Jockstrap Central always seems to have some kind of sale going, plus you get coupon codes with your orders (at least, I did with my first one -- haven't ordered again yet). They also have several inexpensive jocks, swimmer jocks...all kinds of jocks.
Another option you can consider is a cock sock which you can make yourself pretty easily. I know there's instructions somewhere or another online.
If you want as little movement as possible, I'd highly recommend the jocks. You can wear them under boxer briefs or whatever else as well if you'd like. Keeps the packer nice and secured however you wish to arrange it, so no need for adjusting either unless you want to.