It's a good thing that you got him out before he could tempt you or taunt you further. A lot of people will chalk it up to self-doubt, but the honest truth is that he's out there, waiting to prey on you at your weakest. I'll admit I'm at the beginning of my road and I'm already being hit by him.
I'm glad to hear you kicked him out on his rear though, it's heartening to hear stories of victories such as this.