4

What if when taking a test you could select multiple answers and the percent per question you got was (100)(1/n), n being the amount of answers selected (only award points if any a selected answer was correct, and only for multiple choice questions). What this does is give people the option to sacrifice a chance at more points, and take a better chance at getting less points. What are some pros and cons of this system, would it be in any way worth trying?

Addition: If you think this question fits better on a different site, please note. I have no idea what tag(s) to use, and I don’t know how well this question fits in Academia.

  • Comments are not for extended discussion; this conversation has been moved to chat. Please read this FAQ before posting another comment. And please be civil in the discussion, as is expected in a professional environment: unfriendly comments will be certainly deleted without notice. – Massimo Ortolano Mar 1 at 19:27
8

You are not the first person to think about this idea. People use similar ideas in forecasting and your idea is similar to the Brier score.

The premise of this foresting is to be the best at predicting the future. People weight their certainty of predictions over time (e.g., the probability of rain tomorrow is somewhere between 0 and 100%).

Over the short-term, the Brier score penalizes certain wrong answers (e.g., if you say there is a 95% chance of rain but it does not rain, your score is lower than if said there was a 55% chance of rain). But, over the long term, this method penalizes both uncertainty and inaccuracy (e.g., if I consistently make right predictions with 55% probability, I will have a lower score than if I make correct predictions with a 75% probability).

The Good Judgment Project uses Brier scores to evaluate people's ability to forecast and would be a good starting place for you to you look. That being said, I don't know how your proposed idea would work on exams unless you wanted your students to study game theory, forecasting, or predictions rather than your subject material. In general, many if not most education experts think multiple choice questions are a bad way to assess learning (e.g., for some of the reasons described in this article).

| improve this answer | |
  • 2
    Now, if the actual subject of the course is game theory or forecasting, I could see some use for this system when used on a small exam. For all other courses and situations, not so much. – Mast Feb 27 at 8:17
  • 1
    Is the final link correct? It links to the same wiki page as earlier, and not to an article – spyr03 Feb 27 at 11:50
  • @spyr03 yes. I've updated the link to theguardian.com/commentisfree/2013/nov/12/…. Thank you for catching that. – Richard Erickson Feb 27 at 13:05
  • 1
    This answer doesn't seem related in any way to the question asked. The domain is completed different. How does this relate to grading of exams? – Buffy Feb 27 at 13:10
5

One major drawback is answering them all gives points while it does not show that the test-taker knows the answer. It also doesn't discriminate between selecting the two best answers out of five and the correct answer and the most incorrect.

A common way to do this sort of thing is (given 5-answer questions), is to take off 1/4 of a point for incorrect answers and award 1 for a correct answer. Therefore, a student that guesses randomly earns 0 points on average, where as normally they would earn 20% of the points.

The SAT did this when I took it, and I use it occasionally when I write tests for a high school science competition, typically when the material is fairly basic.

| improve this answer | |
  • 4
    It makes sense to award -0.5 points for wrong answers for 4-choice questions, so that guessing is disincentivized and it makes sense to answer anything only if you're >50% certain, and otherwise leave the question blank since you genuinely don't know. – Peteris Feb 27 at 9:38
  • 2
    @Peteris that assumes every wrong answer is random, which is probably not true, so a penalty of -0.5 seems extreme in my opinion. – Azor Ahai -- he him Feb 27 at 17:15
  • One of the principles of good multiple choice design is that all choices must be plausible. – Buffy Feb 27 at 20:54
  • @Buffy Sure, but I've never felt like I've been able to write four equally plausible alternate answers. Maybe it's easier to do that in some fields. You can also include answers that are obviously wrong (or fall out of a common mistake) to identify problem areas. Depends on the goal of the test. – Azor Ahai -- he him Feb 27 at 21:13
  • 1
    @Buffy Yes, like I said, I often write tests for competitions. For those, I don't care what misconceptions they have, since I'm not teaching them. – Azor Ahai -- he him Feb 28 at 1:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.