0

Especially where there are numerous possible answers to each question, e.g. 5.

If one doesn’t know the answer to a question but puts forth a (somewhat) educated guess, at what point is the chance of getting >0 for this question secondary to lowering the value of ‘definitely’ correct answers?

In other words, by how much does ‘indiscriminate’ guessing devalue points accrued from confident answers?

Perhaps this is more of a statistical question, but hopefully there are some general opinions on this matter.

15
  • 18
    Depends on the the scoring/weighting system. Jun 5, 2022 at 6:29
  • 1
    As @DanielR.Collins said. And in addition it also depends on what an "educated guess" means in statistical terms. Moreover, whether guessing is really advantageous for you also depends on your personal preference on how to weigh higher expected outcome against higher standard deviation of the outcome. Jun 5, 2022 at 7:00
  • 1
    To the close voters - which "community-specific guideline" does this question violate?
    – xLeitix
    Jun 5, 2022 at 7:37
  • 12
    @xLeitix: I voted to close because it's essentially a question about probability theory, not about academia. Jun 5, 2022 at 9:02
  • 2
    I would add that the answers all assume that every wrong answer is given the same number of negative points. I have set quizzes where one wrong answer is more reasonable and gets fewer negative points assigned to it. I see no reason a quiz cannot result in negative overall points (see the show QI), other than it would be cruel and unusual. Jun 6, 2022 at 20:46

3 Answers 3

9

Before answering a question, calculate your expected value. If it's positive then you should guess, otherwise you should not.

Expected value is how much you expect to get from the guess. It's calculated by multiplying p(correct) * points for correct + p(incorrect) * points for incorrect. So for example:

  1. If you are 100% sure an answer is correct, then p(correct) = 1 and p(incorrect) = 0. You should always guess.
  2. If you are 25% sure an answer is correct (as is typical for a 4-option MCQ for which you have no idea), then you need the points for getting the right answer to be at least 3x that of the penalty of getting the incorrect answer. If there's no drawback for guessing wrong then the penalty is 0, and you should always guess.
  3. Similarly, if you are able to eliminate two of the options then p(correct) becomes 0.5. You need the points for getting the right answer to be at least equal that of the penalty of getting the incorrect answer.

I once took an exam which awarded 8 points for getting the right answer and deducted 3 points for an incorrect guess. With these numbers, a 50-50 question is still worth a guess. You'll have to do the calculations for your specific case (and also estimate p(correct)).

4
  • 7
    I'm sorry for repeating myself, but as in the other answer the claim that one should guess if the expected value of the points that one gets from guessing is positive, ignores two important aspects: (1) the target function that one wants to optimise does typically not depend linearly on the number of points, and (2) one has to take the standard deviation into account in addition to the expected value. E.g., in financial applications of decision theory, it is typically assumed that higher standard deviation is only acceptable if it is rewarded by a sufficient increase of the expected value. Jun 5, 2022 at 16:07
  • @JochenGlueck My suspicion is that we envision different types of MC questions, or different circumstances. In the exams I had in mind a typical student would in fact have a fairly linear target function - if they get it right they get a slightly better grade, if they get it wrong a slightly worse one. Of course extreme cases are imaginable, like if you think you have just enough points to pass without answering, then you should probably just skip guessing. But for a student somewhere in the middle of the scale I still feel EV is a fairly good heuristic.
    – xLeitix
    Jun 6, 2022 at 8:00
  • @xLeitix: Yes, the strongest non-linearity certainly occurs close to the pass mark of the exam. But there might but other important non-linearities: When I was I student, it often happened that you got the best possible grade with, say, only 85% of the points (due to a version of "grading-on the curve", combined with low average scores in the course). So between, say, 80% and 100% of the points the grade depended very non-linearly on the number of points. (But admittedly that's all quite theoretical, because in practice you typically have insufficient information for any serious optimisation). Jun 6, 2022 at 8:08
  • @xLeitix: Oh, for the sake of completeness I should add that my previous comment does, of course, not contradict your claim that "for a student somewhere in the middle of the scale I still feel EV is a fairly good heuristic". Jun 6, 2022 at 8:37
6

As Daniel correctly observes, whether or not guessing is a good idea depends on the scoring system, more precisely whether you lose points for wrong answers or not. By doing a (very simple) assessment of your expected (point) value we can distinguish the following cases:

Case 1 - No point loss for wrong answers This is the simple (and probably most common) case - a wrong answer is 0 points, same as not answering the question. It should be fairly obvious that in this exam design there is no reason not to guess. Quite frankly, if there is no punishment for guessing, you should always guess, independently of what else is going and or how much of an idea you have what the answer might be. You can only win.

Case 2 - Wrong answers lose some points An alternative exam design is one where correct answers give a point, and wrong answers lead to some amount of point loss (either a full point or a fraction thereof). Not answering of course neither gains nor loses a point. Now it depends on how sure you are and how much you will lose if you are wrong:

Case 2.1 - Full point loss for wrong answers In the most extreme (realistic) case, a right answer gives as many points as a wrong answer loses. Statistically speaking, it's ideal to guess here if and only if the probability to be right is >50%. So if you are quite, but not 100%, sure you should still take the answer. If you can narrow the options down to 2 but you don't know which it is, it's statistically speaking a wash if you pick one or skip the question. If you think three or more options could be right it's ideal not to pick any of them and move on.

Case 2.2 - Fractional point loss for wrong answers If a wrong answer loses a fraction of the points that a right answer wins, you need to generalise a bit from Case 2.1. It's easiest to analyse this case by considering how many of the possible options you can exclude, and then assess your expected value of randomly guessing between the remaining options. For example, if there are four options, and you can exclude two of them, you really only have two plausible options left. When a wrong answer loses less points than a correct answer gives you, you should still guess. If you cannot exclude anything, you should probably not guess (under reasonable assumption about the test design).

Some further considerations:

If multiple answers are possible: If multiple answers can be correct (e.g., answers A and C are correct, but not B and D) the same principles from above still apply, but the space of possible choices explodes to the product of all possible answers. If there is no point loss for wrong answers it's still always ideal to take a guess, but if a wrong answer loses points it becomes much more unlikely that guessing is ever really the right choice unless you are pretty sure you know what the right answer(s) are.

If "none of the above" is a possible option: Statistically speaking "none of the above" is just one more option to consider in your analysis. That said, in practice this is a bit tricky since you will never be able to exclude this option unless you already know the correct answer for sure (and then you don't need this analysis). So if "none of the above" is a possible answer and wrong answers lose full points, you should only guess if you are pretty sure that one specific answer is right.


As a note for educators looking at this - you may read this answer and (correctly) conclude that the best way to disincentivize guessing is to subtract a full point for wrong answers, combined with having "none of the above" as a possible choice. While this is undoubtedly true, consider what the impact on students is who are not guessing, but who simply made an honest mistake. Essentially, this leads to the awkward situation where a partial but slightly incorrect solution is worse than knowing nothing and skipping a question entirely. If you intend your exam to be a measure of student knowledge, that's not what you want.

For exams with calculations, I found "none of the above" to be a particularly nefarious exam design - this prevents students who know in principle how to do the calculation from sanity-checking for simple calculation errors (did I arrive at one of the possible solutions?). In my opinion, the only reason to pick this design is if you want as many people as possible to do poorly, independently of how much they actually know.

6
  • 1
    I have two objections concerning your analysis in case 2.1: the conclusion about the optimal strategy is only correct if (1) the function that you would like to optimize depends linearly on the number of points you reach, and (2) you are indifferent with respect to risk. Assumption (1) is not correct in many exams, for instance since there is often a cut-off of points below which you fail the exam. Assumption (2) is only correct for students who are not risk averse, i.e. who are willing to accept higher standard deviation of the outcome even if it does not result in higher expected outcome. Jun 5, 2022 at 9:24
  • 1
    I find this interesting in an even wider context - if, say, your exam is 50% of the grade for your course. A student going in with a failing grade may be incentivised to, say, pick more answers at random than a student trying to preserve a good grade - the failing student benefits from a larger standard deviation, if the dice come up in their favour!
    – lupe
    Jun 5, 2022 at 17:40
  • 1
    @lupe: For related reasons, both of these existing answers are wrong. The objective is not to maximize the expected score but to minimize the likelihood of a bad grade. So if I know I have answered exactly enough to get an A, but I am uncertain about the answer to one other question, why on earth would I answer it just to reduce my likelihood of keeping an A?!?!
    – user21820
    Jun 5, 2022 at 18:34
  • 2
    @user21820 In my experience, it's vanishingly rare to have that level of precision when estimating your own performance during an exam. Unless you are a perfect student, there are always questions you're not sure about. Exams also often have short answer or essay portions, about which you basically never have better than an educated guess. In my academic career, this made it very rare to be able to strategize about individual questions beyond naively maximizing expected value of points received.
    – A_S00
    Jun 5, 2022 at 21:41
  • 1
    @A_S00: Firstly, what I said is true regardless of whether a student can or cannot achieve the objective. Secondly, if you are a perfect student, then you do not need to guess. Thirdly, when I was a student I frequently got close or equal to 100% and it was the case that I could sometimes know all the answers except to a few questions. Fourthly, it doesn't matter whether you have an open-ended portion, if you are a risk-averse person and want to keep likelihood of bad outcomes below some threshold. (I didn't mention this fourth point earlier because some people simply can't grasp it.)
    – user21820
    Jun 6, 2022 at 7:29
2

If you are guessing completely at random from n choices, then you have 1/n chance of guessing correctly (if completely at random). That means with probability 1/n you get 1 point, and with probability (n-1)/n you get 0 points. Your expected value therefore is 1/n.

Typically, it's desired that a student with 0 knowledge should get a 0% on an exam, rather than 25% or 20% from guessing. This can be done by penalizing incorrect answers by 1/(n-1): Then your expected penalized value becomes 1*1/n - 1/(n-1) * (n-1)/n = 1/n - 1/n = 0 which is what we want. Another way to think of it is for say 5 choices, on average 1 out 5 guesses will make you gain a point, the other 4 will collectively make you lose a point to balance it out, therefore the penalty is 0.25 per wrong guess.

As soon as you can eliminate even one of the incorrect choices, the expected value of course goes above 0. For example, in the same 5 choice test, if you manage to narrow every question down to 4 possible answers and guess them all, you will end up guessing 1 correcly for every 3 wrong - you get 1 point for the correct, -0.75 for the incorrect, and come out with 0.25 points. The more you narrow it down the better it gets. But then again, if you were able to eliminate even one choice, then it cannot be said that you came in with 0 knowledge. You clearly knew at least enough to eliminate the one choice.

Not the answer you're looking for? Browse other questions tagged .