If required to say ANYTHING, you could always just say you had gynecomastia - abnormal development of large mammary glands in males resulting in breast enlargement.. For which one can have surgery.
Not complete BS, its actually pretty much spot on! Its caused by hormone imbalances, skin and adipose deposits in the body, amongst others - even bodybuilders who use AS can develop it..
I'd try not to lie, that way you don't have to remember who you said what to.. It's purely a matter of stretching the truth to suit your needs, not breaking it completely