Your ears are tuned, so to speak, to hear the main frequencies of the human voice.
These are the loudest in the 1-3 KHz range, if I remember right.
In order to not distort small speakers with the lower frequencies, they use less of them.
Same with the higher ones, but not so much.
Lower frequencies take more strength or loudness in order to hear them.
Like hearing the thumping coming from a sound system, which usually will use a separate amplifier and speaker.
Finding ear buds that can handle lower frequencies will cost you.
iPhones and others usually won't spend the extra $ to give you a good speaker.
They use the same old system that Bell telephone used in the earliest phones, boosting the 1-3 KHz range.
They do have that androgynous quality of sorts, but it's pretty artificial sounding.
If you have an Eq or treble middle and bass controls, take out the bass, half the treble and crank the middle.
That's a good approximation of phones, etc.
But you aren't getting the real range of frequencies that voices carry.
Personally, I find that middle range between male and female (different for everyone) with a little breathy sound in it to be very appealing.
Most people do. It brings a smoothness and soothing feeling to the voice.
Unless your range is really on one end or the other, that helps to define an androgynous voice somewhat.
The key is to use it sparingly, otherwise it becomes distracting. Listen to radio jockey's.
Male or female, the best voices use that technique. Marilyn Monroe used it to the very edge of extreme.
When I hear her, it feels like chocolate milk being pour into your ear.

Some of the best feminine voices have a lower range and usually have either a little rasp or breathy sound.
They have always struck me as that androgynous voice.
Ativan