I'm not sure this is very helpful, but I've lost a lot of weight and I can't say I noticed my breasts getting smaller. My chest size (under bust) definitely has, but over/under difference seems to be much the same. I don't think they are growing anymore either, or at least they don't hurt like they used to.
Are you sure they are actually getting smaller, and you're not being overly sensitive? Are you measuring a difference?
Mine do change noticeably when I start or stop progesterone.