I agree that the boyfriend's comment was transphobic and insensitive (and rude!).
If you were a woman then - and they knew you were a woman - then they were dating a woman. Whether that makes them straight, gay, or bi is up to them, since their orientation isn't defined by their partner, either.

(That's a pet peeve of my wife's; she's bi whether she's with a man, a woman, or a non-binary person.) But the *relationship* was between a man and a woman.
Your boyfriend sounds like a guy I knew in college who pressured his cis girlfriend to get breast implants, because a flat-chested woman might as well be a boy...