I'm sure this is at the very least harassment. To repeat what others have said, make sure someone knows that way if, God forbid, something happens, they authorities will have an idea of who could be responsible.
Yes, they won't consider it a threat, and charges won't be pressed, but just so someone knows.
I've had similar things happen at work, but never said anything because the individuals in question didn't persist.
This guy however, is continuing to harass you, and it would be best to let as many people as possible know what's going on.
Personally, I would've reacted in a much more hostile manner, which probably isn't the best idea, but yeah.