Seems to me like you really hate yourself for a number of reasons. Call it spiritual tosh if you will, but I think that, first of all, you'll need to accept the negative feelings. All of them. Work out the why of it, and give it a place in your identity.
Get some peace with yourself. And even better, stop judging yourself.
After that, you'll be able to get a much clearer view on what causes what. You have to get right and wrong out of it.
For a time, I've been as manly as I could. It was quite hard, and really unforgiving, and left some scars on me that will never be healed. Okay. So that happened. I've learnt from it. And I know now that it was nothing but overcompensation for something I wasn't, and wasn't fit for, either. I've sought for advantages of being a man, and found some. (Physical. Strength, endurance, regeneration rate, reaction speed.) I tried to put them up against the physical advantages of being a woman. (Very little to speak of, really. More resistance versus illness.) On that basis, I forced myself to 'be a man' after the whole violence parade. After all; from the world I came from, a woman would probably not have survived.
But.. I accepted that this was not good enough for me. That my saviour had been my own haunting demon, and now is a time where I need a different kind of saviour. (Not a deity; It's just a manner of speech.

)
It's not to say that my story is like yours; it is most definitely not. For one; I have never had much of a 'sex-drive.' I don't really understand a sex drive, to be honest, though I've never been devoid of sex. I have a very strong intimacy drive, and sex just happens to be a very strong expression of intimacy. Also, our situations are completely different. But I think for everyone, the first step is to accept who and what we are, even if in the face of self-loathing.