I was a little reluctant about the purse too, but then I got hold of a couple of nice ones really cheap, and now I carry one all the time.
My "main" purse is a retro-looking corduroy thing, and my "special occasion" purse is actually a vintage QANTAS airlines bag I got for $15 at a local vintage store.