Me? I'm a reasonably gender-conforming gay guy - I just have a hormone imbalance and some unfortunate congenital defects.

(pulls tongue out of cheek) My objection to the terminology isn't really about me. From the outside, I personally could be described as "gender-variant" by a reasonable observer (regardless of which sex they perceived me as). But there are transsexuals who are not gender-variant in their birth sex, and there are transsexuals who are not gender-variant in their preferred sex. And there are gender-variant people who are not transsexual. And there are even gender-variant people who aren't trans at all (the gay guys I fit in with, for instance, could be described as gender-variant, but are not on the trans spectrum).
I think it just needs to be acknowledge that some of us have a problem with our assigned sex, some of us have a problem with our assigned gender identity, some of us have a problem with the standard gender expression for our assigned gender identity, some of us have a problem with all three, and some of us don't necessarily have a problem so much as a preference for nonconformity.