If you are under 65 chances are it is mostly fat, provided you do not have an endocrine disease.
If you are 65 or older, most male would develop some grandular tissue.
If you are a MTF in HRT, you growing breast will have glandular tissue and fat.
So yeah, to answer you question, the difference between man boobs, and MTF or female boobs, is the presence of glandular tissue (mammary tissue) in the MTF breast.
In addition most females have an aerola 1 3/4 inches in diameter, and their nipple is usually bigger than 15 mm