46

This question Inadmissible theorems in research mentioned a scenario in which a student was not allowed to use a theorem that he knew because it wasn't in the course. I've administered exams in which the instructor said "using X technique, solve for Y", and grades of 0 were assigned when a different, admittedly simpler, technique was used. Students seemed to consider the question to be giving a hint, rather than a requirement.

Students often get angry about this, since they feel they got the right answer. The instructor's goal, though, was to get the students to prove that they could use the particular technique, and therefore the answer itself was not relevant.

To complicate matters, there are often times in which not using technique X results in a simpler method for the particular question. This is usually because X is a more general method and is only really powerful when the problem exceeds what is reasonable on an exam. A student could reasonably walk away from the exam thinking that X is just "more complicated" and not worth the trouble.

How can you ask a question like that that tests their ability to use a particular technique in a way that gets them to understand the purpose of the question? Especially, how can such a question be asked to avoid having students erroneously discount the importance of technique X because a simpler method exists for the particular problem?

  • 6
    It seems like the context of the lesson should make it clear that you're trying to teach a particular technique, and the exam is obviously about that. IMO, any student who doesn't see the link is being deliberately obtuse. – Barmar Aug 30 '18 at 15:26
  • 14
    I usually add in bolded text "no points will be given is the technique XXX is used"... – Nick S Aug 30 '18 at 17:13
  • 5
    This is very common, perhaps even universal, in teaching a first course in calculus. Students will be asked to use the definition of the derivative as a limit to evaluate the derivative of some function. Later in the course, and sometimes earlier in the course, students will learn all sorts of short-cut rules for derivatives (power rule, product rule, quotient rule, derivatives of trig. functions, etc.), but these rules are not to be used when evaluating a derivative using its definition as a limit. There are probably at least 5 questions of this very type everyday in Math StackExchange. – Dave L Renfro Aug 30 '18 at 17:36
  • 5
    I'd encourage you to give at least a little credit for using other techniques instead. If you want me to solve a web equation using Spider-Man's theorem and I do it some other way, at least I've shown I can web equations, which is surely worth, say, a quarter of the marks. – David Richerby Aug 30 '18 at 17:46
  • 11
    "I expect you to be able to read and follow directions. If the direction says to do X, then do X." – Daniel R. Collins Aug 30 '18 at 18:54

10 Answers 10

89

You say, "Solve for Y. You must use X technique to demonstrate that you know how to use it, even though there may be simpler other ways to solve this particular problem."

Edit: One should cover in class why the students need to know X. The point in the exam question is to make it clear that using X is a requirement of the question, not a hint.

Example: "Use a Karnaugh map to simplify the following Boolean expression." Here, the question asks, "use a Karnaugh map," not "simplify the expression." The latter is the mechanism to demonstrate the former.

  • 9
    @MichaelStachowsky OK... at the expense of making the question longer, which I try to avoid, you could add "... because there is a class of problems that can only be solved using technique X." I'd rather do that in class prior to the exam: "You will be asked to use technique X on the upcoming exam. I'm going to give you a problem that can be solved in reasonable time on the exam, but the point is to demonstrate X because there is a class of problems that can only be solved with X, so no other approach will do." – Bob Brown Aug 30 '18 at 14:22
  • 12
    I would add something like "Answers using other methods (here you could name them) will receive 0" – Azor Ahai Aug 30 '18 at 17:10
  • 1
    @BobBrown If there is indeed a class of problems that can only be solved using technique X, why not just ask one of those problems on the exam? – JeffE Aug 30 '18 at 17:51
  • 5
    I feel this was a better answer with the "Demonstrate the use of" verbiage in the example. It made the requirement on which technique to use much more pronounced and not just a suggestion. The simplified version is essentially the same as what was used in the OP that was interpreted that the solution was what was important. – Mr.Mindor Aug 30 '18 at 19:42
  • 4
    @JeffE Often problems that can only be solved with a particular, possibly complex, technique are themselves too complex for an exam of reasonable length. So, one offers a "toy" problem but requires the demonstration of the technique, In the Karnaugh map example, an expression suitable for a 75 minute exam with many questions could probably (also) be simplified by inspection. – Bob Brown Aug 30 '18 at 21:22
29

I second Bob Brown's suggestion of writing explicitly in the question that X technique must be used.

As to the concern that they will get a mistaken impression as to the usefulness of X: don't rely on the exam to convey this idea! My philosophy is that an exam is just to assess the student's knowledge and understanding. I don't find that it works well to assign an "interesting" exam problem from which the student is supposed to learn something new. An exam setting is too stressful to be a good time to acquire new knowledge.

Presumably there are examples that illustrate the full power of technique X, where simpler techniques don't work well. You can still have the students work those examples; just not on an exam. Maybe as a homework assignment, or a term project, or whatever. Hopefully by the end of the course, they've done enough of them that one "toy" example on the exam is not going to distort their impression of the technique's power.

(If your educational system uses exams as the sole graded element of the class, so that you can't realistically assign projects and such - my condolences. I don't have any brilliant ideas in that case.)

  • 3
    Upvoted especially because of the parenthetical comment at the end. Even in that case, you can still tell students why X is important, that they will see it on the exam, and that using X will be a requirement. – Bob Brown Aug 30 '18 at 14:35
  • 12
    My biggest early stumbling block as a teacher was falling prey to my desire to put "interesting" questions on exams. I wish I had learned (or been taught) to apply the KISS (keep it simple, stupid) philosophy to exam design sooner... – zibadawa timmy Aug 30 '18 at 15:34
  • 1
    @zibadawatimmy As teachers, we tend to have been better students than many of our student peers. When I became a teacher, I was somewhat aghast at the low level of understanding of some students. However, one learns (still!) to pitch some parts of the course directly to those students; the trouble is to keep the top cohort engaged too. Sometimes peer-mentoring can work in this situation. – Peter K. Aug 31 '18 at 12:03
19

Especially, how can such a question be asked to avoid having students erroneously discount the importance of technique X because a simpler method exists for the particular problem?

I think you're looking at it the wrong way. If you want students to use a particular technique to solve a problem, then I think it should be your responsibility to design a problem where this is the best solution. Asking your students to use a (self-admitted) suboptimal technique to solve a problem is convoluted and goes against any problem-solving skill they might have developed.

Also, if you want your students to understand the importance of technique X, then you must show them problems where it really is the best solution.

  • 4
    I fully agree, but sometimes it isn't up to me. I'm supposed to make the exam according to the instructions I'm given. I've brought up things like this with the instructor, and I get the reasoning behind it, even though I really dislike the idea of requiring a sub-optimal solution – Michael Stachowsky Aug 30 '18 at 17:18
  • 7
    I have to say I disagree - at least on this level of generality. Assigning "solve problem P using method X" and "solve problem P using method Y" (not necessarily at the same time in the semester) is a great way to highlight differences between methods. – Anyon Aug 30 '18 at 17:18
  • 1
    I generally agree that the burden falls to the instructor, or whomever conceives of the test question, to craft a question for which the intended method IS the optimal method (or only feasible method). Students who do not initially try to use the intended method will quickly realize that other techniques are cumbersome/infeasible and they must devise a new approach. Admittedly it does take a bit of time to create such exam questions :) – ChocolateAndCheese Aug 30 '18 at 17:26
  • 4
    @ChocolateAndCheese "Students who do not initially try to use the intended method..." back in the day when I was doing exams (decades before the current obsession with "bite-sized spoonfeeding" rather than "education") a common style of multi-part question had one or two "easy" parts followed by "using the above results, or otherwise, <do something>" - and it was often not trivial to see how to use the earlier results, though using them usually led to a very short correct answer one you did "see the light". – alephzero Aug 30 '18 at 20:46
  • 1
    @MichaelStachowsky: You're supposed to write the exam according to your principles of academic integrity and the students' significant legitimate interest. If the rules contradict this, you should disregard the rules and notify whoever relevant that you acted this way for reasons A B and C. – einpoklum - reinstate Monica Aug 31 '18 at 21:24
6

There are good answers already, but here is a another solution. It may not be the best solution and it may be not practical in some situations, but it may still be a solution. Ask to solve the same problem with two different methods, i.e. write the problem as

  1. (x+y points) Solve the problem A

    a) with method X (x points)

    b) with method Y (y points).

(Still somebody may come up with method Z, but my guess would be that this formulation will make it more clear that you require the use of specific methods. Also it should be simpler to argue that the problem statement was clear if a student with method Z comes complaining.)

5

While, as others have already pointed out, it is important to make the criteria of a question explicit (e.g. words like "must" or "required" are a good bet), I believe that there's a bigger underlying problem, namely a disconnect in expectations, that cannot be solved by wording alone.

The issue

The task "demonstrate knowledge of a specific technique", where the technique is not an optimal fit for the problem, is quite uncommon outside of exam-like situations. Many students will not have encountered this kind of question very often. It is also counterintuitive to anyone already used to working in the problem domain, as they are usually more aware of what method is appropriate under specific circumstances. *

In addition, outside of trivial problems, there's always more than one way to use a tool, and whether or not two approaches are considered equivalent may depend on the level of abstraction (and experience). Personally, as a student with extensive prior knowledge across different fields, I used to have a hard time judging whether an instructor was

  • aware of a simpler alternative solution, but intending to introduce it later
  • aware of an alternative solution, but ignoring it to highlight a specific technique
  • not aware, but open to unexpected approaches
  • not aware, and will dismiss anything but the expected solution
  • expecting a variant of an approach I wasn't even thinking of (e.g. same concept, different notation)
  • just copy-pasting a question from a textbook without thoroughly examining it

More often than not, I ended up answering the question exactly as stated only to find there were subtle differences in interpretation, resulting in anything from zero to full (sometimes +bonus) points.

Add time constraints, ambiguous wording and the potential for errors or "trick" questions into the mix, and it's not surprising that a student might ignore what they perceive as a hint and solve the problem in the most efficient way they know.

Conclusions

While there may be good reasons to design an exam around a "suboptimal" approach, you should not expect a student to be automatically familiar with your course's context, philosophy and didactic goals.

More specifically, don't expect students to know what piece of knowledge you're trying to assess. A question defined in, say, two or three sentences is bound to infer a lot of context, and accurately limiting the scope is a difficult task.

If you give a list of techniques that are not allowed, expect at least one student to come up with something you didn't even consider. If you teach a particular variant of a method, or a particular notation, expect at least one student to have learned the method from a different source and do it that way.

Recommendations

  • Do make your question as clear as possible, but don't expect phrasing alone to solve the issue entirely.

  • Consider publishing a short guideline on question intent and scope that addresses frequent misunderstandings. Include pointers on notation and allowed axioms. Make it general or exam specific, as needed. The point is to move useful "boilerplate" info and caveats out of the text of each question, so questions can remain concise, but will be less ambiguous.

  • When possible for the question, give students some way to quickly assess whether they're "on the right track". E.g. "A correct answer will have the following form/characteristics..."


*While restrictions like this can sometimes be encountered in research, as previously commented in the "Inadmissible theorems" question, these tend to be self-imposed and can be traced to a specific goal.

4

If the exam question states to solve a problem using a particular method, then a mark of 0 for not using that method is fair. The question is to assess knowledge. The marker cannot assess whether a student knows the stated method if they do not apply it. The student cannot (should not) arrive at the answer a different way and still expect full marks.

An exam asks to solve a problem using a particular method.
Use the stated method and get the right answer: full marks.
Use the stated method but make a mistake and get the wrong answer (by propagating the error through): partial marks.
Use a method other than the stated method: 0 marks.

Some students also think that if they solve a problem by:

Problem
--- a bit of working ---
--- some magic happens here ---
--- a bit of working ---
Right answer

that they should get full marks.

  • Students might not know what what piece of knowledge you're trying to assess exactly. The scope of a question is naturally much more well known to the asker than the student. The way I read the question, it isn't about whether the grading practice is fair, but about how to make that scope more obvious. – Ruther Rendommeleigh Aug 31 '18 at 12:19
  • 1
    If a question asks, "Solve the following ODE using the method of undetermined coefficients:" it is explicit that they should use the method of undetermined coefficients. If they instead choose another method they really have no cause to complain about failing on the question. – Mick Aug 31 '18 at 12:25
  • So your answer boils down to "there is no problem"? – Ruther Rendommeleigh Aug 31 '18 at 12:39
3

Here is one way to scaffold the problem:

a. Describe technique X.

b. Use technique X to solve problem Y.

So students can earn credit knowing the technique in general, and they get a hint that they are expected to use it in this problem.

2

If you want to keep it simple, use the word "only"

"using only the X technique, solve for Y"

As others have said, you just need to be clear as to what they use. If you chose to spell out why then that is at your discretion.

If you say use only the x technique, and they don't, then there can be no argument that the answer did not address the question, whether or not the answer is correct.

Personally, I would put a little more explanation in there too to set context though, something like,

"due to the parameters of Z which limit ABC, using only the X technique, solve for Y"

1

There are two options to address the issue here. One already mentioned is to instruct the students not to use other tools than required.

If the forbidden theorem applies only for a limited set of cases while the requested one applies for broader set you can build the test so the workaround gets tricky, hard or impossible to use.

You can also request them to prove the chosen technique can be used and why it is better than the requested one. Correct answer shows the student's knowledge of both techniques.

-1

Sometimes, setting the stage can help.

If your concerned that students won't follow the directions correctly, you can always give the old classic quiz. The one that starts:

Read all directions before starting the quiz.

Then, in the directions, tell them:

Make no marks on the paper besides your name.

So, anyone who does anything beyond putting their name on the paper fails the quiz. Make the grade trivial (or, give a small amount of extra credit to those who pass and take no points from those who fail).

The point would be that the directions on your quizzes and tests are important.

It may be overkill, but it might actually get the point across.

protected by ff524 Aug 31 '18 at 16:07

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.