I've seen that some math teachers design tests which punish errors with negative points.
Why do they assign negative points? What are some pedagogical reasons why teachers might do this?
Grade that goes below zero doesn't make sense.
What makes you say that? In some real-world scenarios, thinking you know the answer and being wrong is worse than realizing you don't know. For example, I'd rather have my doctor or lawyer recognize when something goes beyond their expertise, so I can consult a specialist rather than following mistaken advice. I imagine the same is true for most professionals, such as engineers.
If not answering at all yields a grade of zero, then it's reasonable to award negative points for a truly bad answer. (Of course an insightful but flawed answer may still deserve a positive score, just not as high as the correct answer.)
In practice the most common case I've seen negative scores used is multiple choice exams, for the reason given by GeneMachine, but one can make a philosophical case for applying them much more broadly.
To give an (imperfect) analogy, on Stack Exchange posts can get negative points, thus making the poster lose reputation. The reason is to prevent users from posting low-quality posts in the hopes of getting a few upvotes. Downvotes force the user to only post if he's confident it's a good idea.
Similarly, giving points for correct answers on a test, while ignoring incorrect answers, encourages random guessing. Taking away points for wrong answers forces the students to be sure they really know the answer.
It's to discourage guessing, and to avoid mark inflation.
There are many discussions of negative marking available; here is one:
There are many positives and drawbacks to negative grading, even for free response tests. It encourages academic honesty and self assessment which are important for learning and discourages "BS" answers where the student knows the answer is wrong but is trying to confuse the grader into awarding some points. Some view "BS" answers as cheating. Unfortunately, negative points for wrong answers can punish students who are under-confident, and choose not to write an answer when they actually can demonstrate some understanding. It can also reward students for not even showing up to the exam if the test is hard enough.
I see two parts in this question,
Why negative points? Grade that goes below zero doesn't make sense.
Negative Points can make sense, as a punishment. Grades below zero not so much IMO.
I have a lecturer, who gives us assignments before the exam, we hand them in and get some points for it. You then start your exam with that score, but if you fail a question you get negative points. This is for the reason that the exam only tests for topics we dealt with in the assignments. But you don't drop below zero.
This method prevents people from coping assignments from other students beforehand.
newGrade = sqrt(grade) + grade
. This student therefore had the dubious honour of being the first student in university history to receive a complex number as his final grade.