I remember that soundbit from another thread, I believe? Did you post this already?
I would agree that it is somewhat androgynous, to a large degree probably because of the raspiness and your voice use (inflection, speech pattern, resonance). My assumption would be that if people see you , they will get it right, on the phone, I would guess that sometimes people will be a bit confused.
What surgeries did you get so far?
The next step would probably be not to do another surgery but to do voice rehab and speech therapy. I don't know your insurance situation, but possibly if your voice is raspy and sounds a bit broken (like when you say "Janice Joplin"), you may possibly be eligible for regular speech rehab to get rid of the hoarseness and raspiness? In addition to that, it may be worth looking for gender voice therapy to work on the resonance (and possibly speech patterns).
I had glottoplasty with Dr Kim and no further surgery there and I found that my voice reacts much different to a female resonance and speech pattern than if I was to use a male resonance or speech pattern. The latter makes me sound hoarse and broken, apparently my voice just is not made anymore for this sort of voice use, so it sounds bad. If this is happening to you, maybe you need to "find your voice" by getting into the right resonance and phonation pattern...