There are plenty "middle of the road" clothing choices. I have Nike Pegasus shoes in a women's size 12. I wear a men's 10. I'm the only one that know they are really women's shoes and that makes a difference to me. I also wear men's Jockey Elance underwear. They have no fly front and there is very little difference between the men's and women's. I wear both interchangeably. I have graduated into Jockey Promise Tactel no panty line panties now.
I also have black stretchy t-shirts that are similar to a camisole. They are a men's item, but feel much like something a woman would wear. I have pretty large boobs and I get a bit of perverse pleasure showing them off in these t-shirts.
My summertime "guy-dyke" outfit of khaki shorts, polo shirt and women's tennis shoes is about as androgynous as you can get.
I sleep in a nightshirt I got from Vermont Country Store. I have two of them. They fall about mid-thigh and I've seen many women's dresses that are similar.... the classic "little black dress".
Many women today wear long "tops" that fall down to the upper thigh, almost like a short dress with tights or pants. Try wearing a long man's shirt un-tucked, and then ease into a woman's top that is very similar.
Women have been "transgressing" toward men's clothing for 70 years or so, since Lauren Bacall first appeared in pants.
The "plausible deniability" of men's clothes that only LOOK like women's clothes can provide some cover that could make you feel more secure.
Best Wishes,
Randi