TSOTL I never took to be transphobic, since Harris has Clarice and Lecter discuss how transsexuals are rarely involved in violent crimes and then Lecter goes on to explain how Buffalo Bill isn't an actual transsexual. He was rejected from the gender clinics for not meeting the criteria for successful diagnosis, but is afflicted with some trauma and/or disorder of the personality in which he would like to be a transsexual - to escape his general identity (not his gender identity). "He's tried to be a lot of things, I expect," is what he knowingly follows this up with, meaning Bill was searching for ways to escape being what he is, or to feel special, but will ultimately never succeed.
If there was some insinuation within the book or the film that this is what transsexuals do - that they are all totally nuts or serial killers, that would be pretty transphobic. But I felt the film and the book did distance Bill from real transsexualism. Those who thought the film didn't when it came out either weren't paying attention to the lines, or else thought that any movie with the antagonist having any connection at all to the topic of transsexualism would harm transsexuals. I don't think it does. I also think it's an excellent film.