4

Our multiple choice exam was clearly too difficult.

  • N questions.
  • N + 10 total duration (in minutes).
  • All questions have the same value (100 / N).
  • Wrong answers have a penalty of 25% (25 / N).
  • Only one correct answer per question.

What strategies do you recommend to consider reviewing the grades?

What we though of:

  • Remove the penalty for wrong answers.
  • For each student, increase the value of each correct answer.
  • Add X (e.g. 10 points) to each final result.
  • Add X% (e.g. 10%) to each final result.
  • Remove the questions with worst performance.

We can combine options and each one has advantages and drawbacks. Note that we use absolute grading (not grading on the curve).

What is your experience?

Follow-up: Thank you all for the feedback. In the end, we decided to review each question. In some cases (four in total), we decided to consider as correct some of the incorrect options that were not 100% clear (and had a significant number of students choosing them). Plus, we added a bonus of +1 points to all students. We kept the penalty for wrong answers since it would be unfair to remove it.

  • 2
    Please add more detail. Was the duration the problem (did many students not have enough time to finish?) (See What can a professor do about an exam that was too long for the allotted time, after the students have completed it? if so.) Were there specific questions that many students got wrong, suggesting that the question may have been unclear? Were there specific topics that students seemed to have a weaker grasp on? Something else that gives you insight into what went wrong? – ff524 May 3 at 15:26
  • 1
    Then the exam can't have been so hard after all. ;-) – Karl May 3 at 22:36
  • 2
    Are there questions where there are no more correct answers than would be expected by random chance? – Dawn May 3 at 23:40
  • 3
    I have no idea what "we use absolute grading (not grading on the curve)" means when one of the options is to add 10% to everyone's grade. – Pete L. Clark May 6 at 15:43
  • 1
    To the follow-up: how is giving bonus points any different from curving? – Kimball May 7 at 15:47
8

There isn't enough information here to really give good advice. You suggest the exam was too difficult. It may also be that it was invalid. It may be that some of the questions were stated in such a way as to be misleading. There are measures of the validity of such exams, by the way. They measure the validity of a question by the proportion of students who answered it incorrectly compared to how those same students did overall. Some questions are negatively correlated with overall performance.

Another issue is what your institution permits. Some have very strict rules about this. They are IMO unwarranted and counterproductive, but they may bind you.

But absent such rules, you should care more about fairness than you do about numbers. To achieve fairness you may need to drop the exam or give an alternate. You may even want to rethink your overall grading scheme.

One simple modification overall is to give course grades based on the, say, 8 best of 10 assignments/exams/whatever. Then the question of a poorly designed quiz never arises. Another way is to have quizzes every day so that no individual quiz is determinative of much of anything.

However, some of the things that you might try will leave some students unsatisfied; especially the best students who worked the hardest. You can be kind to the strugglers, but not at the expense of the superstars. The situation is worst if the system itself puts the students in competition with one another for grades. Strict curve grading is IMO immoral as it makes it into a zero sum game where I can only advance at someone else's expense. If the system doesn't permit top marks for everyone (assuming it is deserved) then it is fatally flawed.

Your purpose, I hope, is teaching, not grading. Use the exam as a teaching moment. Even have a class discussion about the questions that caused difficulty. Try to learn why people did poorly. Even permit different students to have different sorts of adjustments as needed.

I used to have fairly strict rules about such things, but the understanding was always "This is the standard and you will do no worse than X if you do Y". And I always tried to make it an advantage to learn something even if was after the deadline or the exam. Make people want to learn, not just want to maximize points.

  • Thank you for your advice. I fully agree with "fairness is the most important issue now". We are quite confident that the exam hadn't design issues. Some of the questions were problematic (i.e. hard to understand, required very specific knowledge, etc). But removing them is not fair for some students. – Anonymous Professor May 3 at 14:58
  • My goal was to discuss 'when everything fails and you have no alternatives, what is the fairest way to re-grade the exam'? – Anonymous Professor May 3 at 15:06
  • 4
    @AnonymousProfessor when everything fails implies the exam instrument itself is worthless (e.g., you gave your Arabic language class a test on the Mandarin language). You should discard it and create a new one. – emory May 3 at 17:16
4

You're overanalyzing this. You just need to curve the results. Decide what you think was an A performance (perhaps a really low score if the exam was that tough) and what was a C and then interpolate. Problem solved.

  • Thanks but we use absolute grading (not grading on the curve), thus this is not possible. – Anonymous Professor May 6 at 8:40
  • 3
    That was dumb. Now you know why most instructors don't do that. But you should have mentioned that constraint in your question. – Nicole Hamilton May 6 at 10:45
  • 2
    To be fair, OP is talking about giving everyone free points because the average exam score was low -- thus, the class is "effectively" curved after all – cag51 May 6 at 15:34
  • 1
    What universities have policies requiring absolute grading? I suppose it's possible there are places where this is the common, but required by "policy"? Academia is a place where instructors usually get wide latitude to conduct their classes any way they like, So, I find this claim of "policy" to be pretty suspect. – Nicole Hamilton May 7 at 17:06
  • 1
    @AnonymousProfessor Absolute grading is a dumb policy for exactly the reason you observed: There are a lot more students than instructors so it's lot more likely that one class will have the same mix of abilities and performance as the the next than that instructors will always create equally difficult exams. It's basic statistical sampling theory. – Nicole Hamilton May 7 at 17:09
4

What we though of:

  • (1) Remove the penalty for wrong answers.

  • (2) For each student, increase the value of each correct answer.

  • (3) Add X (e.g. 10 points) to each final result.

  • (4) Add X% (e.g. 10%) to each final result.

  • (5) Remove the questions with worst performance.

In my view, you must avoid any alteration that changes the relative value of questions or answers from the marks listed on the exam. This rules out options (1), (2) and (5) in the above list. If you were to use any of these options, it would disadvantage students who did well on the questions whose relative marks are reduced, or students who declined to answer a question based on the relative penalty for a wrong answer compared to a correct answer. Such alterations are unfair to those students and would be grounds for legitimate complaint and appeal of marks.

The only fair way to "regrade" an exam that was excessively difficult is to scale all the marks up with a simple positive affine scaling of the marks. This could entail a flat increase in the mark of each student, or a percentage increase, or any transformation of marks according to a positive affine function. This method of scaling preserves the relative value of all questions and answers in the exam, and preserves the relative marks of the students.

  • "This could entail a flat increase in the mark of each student, or a percentage increase" Well, no. A linear scaling is always proportional. A "flat increase" is a violation of linearity. I suspect you meant to recommend an affine transformation (y=mx+b) not a linear one (y=cx). – Ben Voigt May 4 at 21:00
  • 2
    I'm not sure why the scaling needs to be linear (or affine). – Andreas Blass May 5 at 4:03
  • 4
    Agree. Students will have strategized the use of their resources to meet the stated marking scheme. What OP describes is already sub-optimal: very hard exams tend to penalize students who know the material but aren't good at doing this strategizing. But changing the stated marking scheme after the fact also penalizes those who did do a good job strategizing. Concrete example: students who didn't answer questions to avoid the wrong-answer penalty will be enraged if you don't impose such a penalty -- they could have gotten "free points" by just guessing randomly, but you misled them about this. – cag51 May 5 at 20:04
1

If you simply reduce the total points, say grade as a percentage of 80 rather than out of 100, it essentially converts whatever questions they want into a potential 20 bonus points. Personally I like this better than free points, as it increasingly rewards students that do better, rather than being a freebie for all which compresses the scores together.

Dropping questions completely is a bad idea as others noted. But you can convert them to bonus questions. If questions are ambiguous though, you have little choice but to accept multiple answers. That's a separate issue.

As for my experience, if you seem to be too flexible on grading, you may encourage a lot of begging and whining. Pick how you want to curve them and present this as the law.

  • +1 for the last paragraph in particular: students react well to free points, even though linear transformations rarely affect outcomes in classes that are curved (or effectively curved), but react poorly to uncertainty – cag51 May 6 at 1:11
0

In general, any change made to the grading system would affect some people positively and others negatively. Usually, I simulate all the ideas before committing to a new grading scheme. Although, in case of multiple choice exams, I would be more cautious as some of your exams rules influence how people answer questions (e.g., when I was a student, to avoid the penalty I usually refrained from answering questions, which I was unsure about their correct answer). If I was in your shoes, I would consider the following options (sorted by my preference):

  1. Give an optional alternate exam for those who are not satisfied with their grade (this would be forbidden in some cases, in particular for the case of the final exam).
  2. Give parts of this exam weight to a future one.
  • 1
    "any change made to the grading system would affect some people positively and others negatively" - although this may be true for a zero-sum game where students are viewed as in competition with each other, it would also be straightforward to assign a grade max(MethodA, MethodB, MethodC) for each grading scheme, which would be especially appropriate if some questions are excluded from some exams but other students got them correct. – Bryan Krause May 3 at 16:55
  • @bryan-krause: I don't see the situation as a zero-sum game. Let's assume that the instructor decides to go with omitting some of the hardest questions (based on the number of wrong answers). In this case, students who have put time and effort into those questions and got them right, would not benefit and even suffer from it. Regarding your suggestion, I agree as it could solve the problem I mentioned. – Ehsan May 3 at 17:18
  • If we give X extra points to all students, it doesn't seem to affect negatively any of them. – Anonymous Professor May 3 at 21:26
  • 1
    @Ehsan: In some environments, GPA means nothing and class rank is everything. Those are always zero-sum games no matter how careful you are with grade transformations. – Ben Voigt May 4 at 21:04
0

As a clarification: Were unanswered questions exempt from the penalty? If so, it becomes unfair to remove the penalty, as people leaving blank questions might be a conscient choice.

Removing questions with worst performance might also be unfair to students who actually knew how to answer them, unless these questions were really ambiguous ir flawed (such that there isn't actually a right answer).

The mathematically complex, but for all other aspects lazy method is to curve all grades.

You compute the mean grade (let's say 30) and their standard deviation (let's say 10). Then you pick a new mean and a new deviation that suits your taste (lets say 55 and 15). I.e. pick higher mean to have more students passing, pick higher deviation to have more discrepancy between notes.

For each student (start with the lowest grade), re-compute the grade by matching a gaussian cumulative distribution function, in excel see this reference, the new grade would be given by:

NORM.INV( NORMDIST(old_grade,30,10,True), 55,  15)

Where I've already replaced the numbers with my example. Or without replacement:

NORM.INV( NORMDIST(old_grade,old_mean,old_deviation,True), new_mean,  new_deviation)

Check if the lowest grade was increased or decreased by this process. You don't want it to decrease.

The idea of this method is that any test can be as difficult or as easy as the professor wants it to be (if the professor is good enough in designing tests). Hence, it's like you wanted students to get low grades on the test, but marked them with average grades in the end. The ranking of grades remains the same. You can fail a given percentage of the students if you want (let's say you knew that getting a 30 was difficult, but a student who got 10 actually knew nothing), you can also let everyone get a passing grade.

I strongly suggest not to abuse this system, and most of all not to use it to lower student's grades. Do it only if a test was very poorly designed.

  • Yes, unanswered questions are exempt from the penalty. Thus, I agree that it wouldn't be fair to all students to remove it. – Anonymous Professor May 3 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.