How to create a grading rubric that properly incentivizes and challenges students

Question

Since I haven't do much TA duty (only 1 semester so far, starting 2nd one), I have struggled to come up with a good way to grade homework. My last grading rubric ended up creating strange incentive pattern wherein people will do all the optional questions, while completely ignore the required questions (or just write some random nonsense to get some extra pity points); and since I posted my rubric publicly, I couldn't change it. Since I am about to grade the first assignment of this semester, I need to come up with a new grading rubric, so I need help with the following optimization problem.

The situation is as follow. Every assignment will contains around 5 required questions, and 1-3 optional questions. Required questions would be very simple and can be solved by understanding the material, while optional questions would usually require new ideas. I have freedom in choosing any 3 required questions to grade (and they have to be the same questions for all students), and anyone optional question to grade (could be different question for each student). I might award completion points questions I do not grade, but since I am not supposed to grade them, I cannot discern whether the problem is actually solved (and given the amount of grading I have to do, I can't really afford to look into more than 4 questions per student).

I want to have a grading rubric that create the incentive such that students are most likely to do all the required questions and at least 1 optional question, subject to the following constraints:

• Doing something extra never hurt the grade (even if "something extra" is a drawing of an goat).

• Solving harder questions get higher score than solving easier question (note: I get to pick the maximum score for each question).

• Maximum possible score can be obtained from solving all required questions only.

• When it comes to required questions, partial credits must be awarded for answers that get certain part of a correct answer.

• For questions with explicitly multiple parts, every part must get the same maximum score, and any solved part must be awarded scores (note that in a multiple parts question, it is very common for part (a) to be much easier than part (f)).

• Anyone who attempted all questions (both required and optional) get at least 50% of the maximum.

Anyone can help me with a good grading rubric?

EDIT: since I can't make comments, I will clarify some of the answers/comments below.

First, about grading randomly comment. That was exactly what I did last semester. Grade random required questions, grade all optional questions (since you know, they are not required, so it would be unfair if some get points and some don't for the same work), and give optional questions much higher maximum points (since they are much tougher). Result: people only do optional questions, since these are the only one with guaranteed points, and give a lot more points.

Second, point #3 makes perfect sense, and I don't see how it is restrictive. After all, other questions are known as optional for a reason: people don't need to do it to get full score. And #6 is what the professor requires of me: the class is supposedly a hard class that is optional for the major and is also regularly attended by people who are not in the major that are simply interested, so he doesn't want to fail anyone who put in "honest effort" (that's the word he used, so I guess 6 goats and 1 turtle won't count, but a bunch of gibberish equations would).

As for the answer below. I won't make the mistake of making the rubric public anymore, the students would have to ask for that in which case I can explain to them. Grading the best looking optional problem will violate constraint #1 here. This is because it is really hard to see which answer are better without and detailed look, since optional problems have lots of intricacies, are long, and tend to have fewer parts. And if I judge wrong, it means someone who attempt 2 problems might end up with a low score because I picked the bad one, compared to someone who only do 1.

Answer 1

In general, I would recommend keeping your grading rubric simple -- complicated rubrics have a way of causing confusion and sometimes resentment when they're misunderstood.

In the situation you describe, I would choose to compute the score as the average of that on the three required problems, weighted equally. If the student solved any of the optional problems, choose whichever looks the best and substitute this score for the lowest of the three required scores, but only if it improves the student's grade.

Answer 2

One suggestion is to do what is common practice at my school (I am a student):

• There is a sheet of required questions, worth (for example) 100 marks, and a sheet of optional questions, worth 50 marks (there are only a few optional questions, but they have high marks for each question

• If the student performed better in the required questions than in the optional questions (eg. 80/100 in required and 30/50 in optional), then the final score is the score for the required questions only (in this case, 80%)

• If the student performed better in the optional questions than in the required questions, then the 2 scores are added together (for example, 60/100 on the required questions and 40/50 on the optional questions, then the final score is 100/150 or 67%).

Answer 3

Do you not have some inconsistency between 'Maximum possible score can be obtained from solving all required questions only.' and that you don't have enough time to actually look at all their answers to required questions?

It seems to me the simplest way to meet most of your points without introducing weird incentives is a two phase approach:

1/ Equal points between the required questions (so 20 points each if 5 questions and going for a total of 100). For these, you actually check 3 each week and give completion points (ie assume correct) the remainder.

This means that someone who answer the three questions you look at correctly and provides something sensible looking for any others can achieve the max points only from required questions.

2/ Optional questions are to recover points lost through a poor answer to a required question and each optional question can recover enough lost points from two required answers. Students give what they think is their best answer first and then the others. You always mark the first optional answer if provided (whichever question it happens to be for that student). If the student has not yet got to full marks, you go through the other optional answers for that student.

Note that (2) runs the risk of answering more than 4 questions per student, but it is likely that some students won't answer any optional and/or the student's pick of best answer will be good enough.