I think purse is a general term, while handbag is more specific. In my mind at least, purse includes anything that had the function of a purse: handbag, shoulder bag, tote bag, messenger bag, etc. while handbag is more limited to bags that are held in the hand.
Anyway, I love purses, because you can just carryit and know you have everything you might ever need in it. Pockets, you have to decide on a day-to-day basis what you're gonna need in your day, and I dunno about you, but for me that's a guarantee I'll forget something important. Besides it's just such an awesome upgrade on what you can carry. However would you carry tissues, moisturizer, an umbrella, lip balm, a wallet, spare hair stuff, a snack and such without one? Seriously, I'd never go out without my purse again, except maybe if I go out strictly for a walk. I feel naked without it.
Not to mention purses are an absolute protection against forgetting things in your pockets and washing them, causing annoyance at best; a disaster laundry load at worst.