Serenity
Reflecting on others' answers and comments, I feel that one more section should be added here, because there are practical things you can do to help the situation, from your side only, with zero direct interaction. Although I would still gather evidence in any case (since it might be of value later should a dispute come to the surface), it is possible for you to improve your experience here by improving yourself. It's easy and natural to focus on the fact that he is the aggressor and by rights he should be changing his behavior, and to forget that, completely independent of whether he stops or not you can choose to become better at not being affected by baseless criticism.
Give yourself a mental picture of a small child attempting to attack an adult who calmly holds himself out of harm's way by virtue of having arms twice as long as his attacker. Your stalker has mastered the art of playing the flailing child--you can grow long arms.
Mentally rearranging your perspective on the whole situation can help. You can say "I am going to end up better for this by learning the valuable skill of ignoring trolls. Think what I would have had to pay a trainer for this, and he's providing me all this learning experience for free." It is, after all, entirely possible that you are going to run into other situations in life where the ability to be calm in the face of baseless or childish criticism helps you out immensely, and this is an opportunity to learn that skill in a relatively risk-free environment (as the stalker really has no actual power over you in this situation).
In short, you have, effectively, been focusing on the question "How can I get him to stop this behavior, without making a fuss?" But you can get an excellent result by instead asking the question "How can I get this behavior to stop bothering me?". The second question can be pretty much entirely in your own hands, and the things you do to work on it will help you in other aspects of your life, even enabling you to help others deal with similar issues, etc.
It's more than just ignoring (which, I think it should be said, is fantastic advice if you can do it--I haven't gotten to that point yet personally)--it's a conscious decision to rise above the behavior of the other party. Rather than "he's attacking me and I feel powerless to stop it without escalations that I am not comfortable getting into" it can be "he's attacking me, but I am using this as an opportunity to practice, and therefore develop the power, to not be affected by this behavior". From the outside it may look the same, but mentally, this kind of approach can make all the difference.
I think this is both the best option and the hardest to execute on. In the times when I have implemented such measures myself, I've found it to be incredibly empowering, and even in some cases ended up with unexpectedly good relationships with people that at one point had occasion (they felt, at the time, at least) to attack me, simply because I consciously decided that I didn't need to fight back. I don't think I've faced exactly the same situation as you are here, but I believe this might help you and sincerely hope it does--best of luck.