For those that have had the sex on their SSA records changed, it may be wise to order a copy of your numident info from SSA.
In my case, surgery was completed Oct 2003, and I took my letter to SSA in Jan 2004 to update the sex on file. The lady at SSA was pretty clueless, but said that all was done and that she had corrected my information. Over the next few years I had a few non specific problems that I suspected were SSA related, but I blew it off.
Recently I've taken on the task of updating my Texas Birth Certificate, and in my document gathering I ordered a copy of my numident. After it arrived, I was floored to find that the lady at SSA only "reconfirmed" my name change in 2004, and never updated the sex on my file. Almost 13 years post and now I realize that SSA still isn't done. Ugh.
Check to ensure that changes are actually made, and that you weren't just lied to by a clerk at SSA, DMV, etc.