Yea, I had the same problem... You need to keep practising, listening, comparing and focusing your whole attention of your recorded voice, because I assume you do it. If you still have problems with judging whether your voice sounds feminine or not - ask somebody
Even stranger! Due to Aspergers, I had another problem. My prosody owing to aspergers was really strange, a bit theatrical, weird... I was considered to express myself very baroque. Now I need to speak carefully, controlling myself, can't express such strong emotions or feelings, need to center on calm speaking, because due to such prosody, you can really sound bad... Or simply can't hold the voice on the right tone, pitch and stuff