I don't. I just resurrected all of the clothes I had to stop wearing when I got too fat and too, uh, top heavy.

But, seriously...when I was younger, I had trouble finding shirts to fit me. Then I started gaining a bit of weight and began acquiring a closet full of shirts in men's small. I mostly got them from Robinson's and Bullock's/Macy's, with a few online forays into Lands' End, Paul Fredrick, and another menswear company whose name escapes me at the moment.
If I keep losing weight, which I plan to, I'll have to either get smalls and have them tailored, or start shopping for extra smalls.
I'm 5' 6 1/2", by the way. Yeah, gotta point out that extra half inch. Because, you know, size DOES matter.