Complicated issues -- sexual arousal from cross dressing is often considered "fetish" by the psychiatric community. The psychiatric community is still trying to figure out how all this "gender stuff" works within the context of a socio-cultural paradigm.
There are no clear, simple answers. Part of what you feel is likely fetish. Part is gender dysphoria, or possibly something like gender dysphoria. No confusion, just not pleased with what you have.
Women's clothing is intended to be exciting. That's the function of female sexuality, to get men excited. There are all sorts of complex sexual/gender messages, signals being conveyed by dress. Women reveal and accentuate physical features in order to attract a mate. Men are conditioned to associate the clothing features with the sexual allure of women. Panties are an obvious example.
Perhaps one internal narrative that arises in the attraction to women's clothing is that it affords a physical connection to sex that is more easily controlled and access than an actual woman. If one becomes the woman of one's sexual desire, that desire comes under one's control. It'd be nice to have access to sex on demand.
The dissonance in the narrative for me is that I get conflicting messages. On the one hand I desire sex with women. On the other hand, I wish to be female. Let's further note that the sex I desire with women is as another women, not as a man.
So there's "sex" -- physical characteristics, like genitals.
There's "orientation" -- who/which gender you're sexually attracted to.
There's "gender" -- who you go to bed AS . . . the ID in your head that is independent of physical sex.
Just to make it problematic, and more interesting, there are several variations of each: of sex, of orientation, and of gender. Fortunately, there's no required registration. You don't need to decide where you fit, and no need to commit to what you haven't yet decided. Never mind what your govt. ID says. They haven't figured it out yet either!
I've been known to "flow" from one "gender" to another in the course of walking down the stairs, putting in earrings, brushing my hair. Fortunately for us and for society the cultural norms are becoming more inclusive, less rigid, less constraining.
