The queen's I've known are proud of the phrase.
But, Drag Queen, Transsexual and Transvestites are all very distinct groups.
If you call me a Drag Queen, I'll probably laugh at you first, and then slap you a few seconds later, because it is a denial of my feminine identity, essentially saying that it's an impersonation.
So, it depends who you're saying it to.
I also know some of them prefer Gender Illusionist, Drag Performer, Female Impersonator...etc...