If you feel comfortable enough with yourself to be able to pack, then go for it. But don't be too down hearted, many members of the gay community understand a lot about transgender people so it would have been easy for him to spot. Many other people wouldn't even take notice if you have a bulge or not, it's not something straight people tend to do.
Good luck. 🙂