Yeah, it sounds like your problem is with being seen as a girlfriend, and you've somehow transferred that onto the word boyfriend, even though - as noted already above - gay men have boyfriends all the time.
Other terms to describe him: significant other (SO), partner, "special" friend (hehe).
Terms to describe the relationship (pick whatever fits): we're in a relationship, together, dating, seeing each other, exclusive, hanging out
Lots of guys use weasel words to avoid the boyfriend/girlfriend label altogether, so he's probably aware of the choice of 'seeing each other' being less than 'boyfriend'
Key here is that you're hurting your boyfriend, because he thinks you are somehow ashamed or unwilling to have him as your boyfriend. Are you out to him? Can you explain the issue? And if not, then I think you need to do your best when people to answer about how happy you are to be together without using the magical word that bothers you. Changing the subject or not answering is a problem.