17

What are the pros and cons of the practice that some questions of a (written) math exam are exactly from the course? For example, asking about the proof of a theorem proven in a course.

The argument I have heard against such practice is that some students tend to just memorize the material, but on the other hand, some people say that the results covered in a standard course are usually essential theoretical results and even memorizing them has some pedagogical benefits.

Edit: The majority are average students are able to do routine exercises, are more and less attentive. Few students are outside average range (very exceptional students or below average). Just a normal class.

9
  • 3
    Describe your students. Jun 18, 2023 at 14:49
  • 2
    "even memorizing them has some pedagogical benefices." Has it? Jun 18, 2023 at 17:25
  • 31
    "Average" is not a useful descriptor. My average undergraduate student is utterly incapable of doing anything they haven't been told explicitly in lecture, and my below average student is genuinely shocked that any human being is capable of such a thing (and genuinely finds recalling what happened in lecture a challenge). Somewhere else, the average student may differ. Jun 18, 2023 at 22:46
  • 8
    @AlexanderWoo Hopefully you haven't explicitly told them how to follow you on Academia.SE. Jun 19, 2023 at 23:14
  • 2
    @AzorAhai-him- I don't hide what I think from my students. They need vastly better problem solving skills in order to succeed after they graduate, and I think it does them a disservice to be learning this by having trouble keeping a job or getting promoted rather than learning this while they are still in university with more resources to work on it (or have a chance to change their studies to an area where their problem solving skills are better.) Jun 28, 2023 at 23:06

9 Answers 9

19

Have a look at Bloom's Taxonomy. Asking students to understand a proof from class well enough to recreate it is near the bottom of the taxonomy. This is perfectly fine because you definitely need the students to be at that level before they can apply their understanding to new situations. But the questions you need to ask yourself are:

  1. how much of my assessments focus on understanding vs on application?
  2. what other questions could I ask to test their understanding?

For a proof-based class, the students should be engaging with higher levels of the taxonomy so you don't want the majority of your questions to be "recreate what I did in class." One or two questions should be ok given some thought about the key concepts in the proofs you expect your students to know.

16

One big advantage of this sort of question is that it tests what it aims to test extremely well. If a student knows the material they will be able to do the question (even on a bad day) and if they don't they won't (even on a good day).

When you ask about something new, there is inevitably some randomness about which students will spot what they need to do differently. A slightly different question might suit different people, and someone having a bad day could miss the key idea even if they would probably get it under better conditions.

Of course, that is also why you need plenty of questions higher up the taxonomy - you need to measure those skills too, and since they are harder to measure you need to put more effort into doing so.

1
  • I would definitely question the claim "If a student knows the material they will be able to do the question (even on a bad day)". Memory is a skill just like any other, and one that is particularly affected by "bad days" (e.g. lack of sleep). I know from personal experience that even top students have days where they have trouble recalling previously shown proofs from the course.
    – Birjolaxew
    Jun 21, 2023 at 13:34
11

Variation in familiarity/difficulty is desirable in exams

An ideal examination will have questions that range in their level of difficulty and familiarity, including some questions that are familiar to students and offer an opportunity for "easy marks". Including questions that have already been asked and answered in lectures or tutorials (or which were assigned as homework) meets the function of giving familiar questions with no nasty "twists" that would require higher-order skills. You should also include some questions that are minor variations on things the students have seen before, some questions that are major variations on things the students have seen before, and also some questions that are totally unfamiliar, but still use the same mathematics that is being taught in the course. This variation will help students get used to adapting to solving new problems with varying levels of familiarity.

There are some areas of applied mathematics where it is useful to memorise a set of equations or results, particularly when what is being memorised is the correspondence of an equation or result with its name. For example, for students doing work in probability/statistics, it is useful to have memorised the density functions of all or most of the core families of probability distributions (e.g., being able to easily identify the normal distribution, gamma distribution, binomial distribution, etc.). This is useful in the same way that it is useful for medical students to memorise the names of parts of the body to aid in learning the substantive parts of medicine.

Aside from these types of cases, one of the key things you will need to impart to your students is that "memorisation" of results that is not backed by understanding is not a method that can lead to long-terms success in mathematics, even over the period of a single course. It is possible that a student could memorise a proof and regurgitate it, but it is not really feasible to pass an entire mathematics course merely by memorisation and regurgitation (and it is certainly not feasible to have a successful mathematics career this way). Memorisation of an entire proof comes with a significant cost in terms of cognitive load. Memorising the steps of a proof you don't understand requires significantly more cognitive load and likelihood of error than merely remembering the core method and main steps of a proof you understand. Moreover, the lack of understanding means that you can only reproduce the same proof and you can't handle variations on the problem if these arise.

In a learning situation, the student who memorises the proof (in lieu of understanding) learns that there is a significant cost to not understanding something and merely reproducing it by rote and that this is an inefficient method over the long-term. At most, memorisation of a proof can offer a minor short-term benefit. It is useful to impart this reality to students explicitly, but they will also learn it as they go when they attempt to "cram" for exams.

1
  • 1
    This was my first thought when reading the question. You want to test everyone at different levels to see who understands the material, who doesn't, and who can extrapolate beyond what's been presented. And also to see who's been attending the class. I took a pretty elementary algebra course, and on the last day of class, the professor said, "You don't really need to know this..." and showed how to multiply matrices. There was an final exam question asking for the result of multiplying two matrices.
    – Wastrel
    Jun 19, 2023 at 15:36
10

My sense of your question is that this is entirely normal and common. But the question isn't (shouldn't be) one that emphasizes memorization, but, rather, insight into how such a proof actually works and contributes to the overall theory. That requires thought, of course.

I'll note that some proofs in math are especially useful when they provide not just proof, but insight into the underlying concepts of a field.

I'll also guess that this is most useful for the important theorems, rather than the corollaries and such.

For this approach, I don't see any downside.

4

Often I do exams like this:

  • First part, lesson questions
    • Question 1) Recall the definition concept X.
    • Question 2) Recall theorem Y.
    • Question 3) Recall the proof of theorem Y.
  • Second part: solve a math problem (divided into several subquestions to guide the steps).
  • Third part: another math problem (usually one of the two problems is much shorted than the other)

Not surprisingly, the students who fail questions 1, 2 or 3 also completely fail both problems, and most students who succeed at questions 1, 2 and 3 mostly succeed at the problems as well, although there can of course be a few mistakes along the way.

So, it looks like these three sets:

  • the students who memorised the lessons
  • the students who understood the proofs
  • the students who are able to apply the concepts to solve a problem

are mostly one set, not three different sets.

The exact score earned in the exam due to the two problems vary a lot between students, due to varying number of mistakes or of how well they handled the time constraint of the exam; but the first part very clearly divides the class between those who have a basic understanding of the lessons and those who don't.

If I had to explain why, I would guess that the concepts are actually easy enough that if you have memorised them, you will not have too much trouble understanding them; but they are complicated enough that there is no hope of understanding them if you haven't even memorised the basic building blocks.

1
  • 6
    I think this is a great observation. I wish it guided more research/pedagogy. My theory for some time is that testing the level-1 knowledge of definitions is under-emphasized in math education. Jun 19, 2023 at 15:04
2

One disadvantage – or at least restriction – is that it doesn't work in an open-book exam.

1

You've received some good answers already, I'll just add an example of what might happen in case you overdo it.

At the uni where I studied there is a lecturer in theoretical physics who was (and maybe still is, I don't know) known among students for basing her exams largely around (minor variations of) exercises that were studied in the exercise groups during the semester, and thus for the exams being easy to pass even when you struggle to understand difficult theoretical material.

And then, of course, more than on average perhaps students that are looking for an easier way to pass the exam attend courses by such a lecturer, for example students that have failed the course before or, interestingly, sometimes even students who take the course one year earlier than scheduled just to get that opportunity (thus not having attained all the knowledge from preceding theoretical physics lectures they might need for the course).

I'm saying nothing about the extent of such an effect and one whether you should put this down as a pro or a con... ;)

0

Okay, so I would like to share my experience working with some professors and also in the creation of test papers for university students in Russia. Some questions on math exams are taken directly from the course material, not just math but everything.

Here are the pros:

It solves the problem of a lack of question materials. You cannot keep creating new questions all the time. In fact, professors often create a list of questions based on what was elaborated on during lectures. And, of course, the lecture program was created based on a "course" book. Even if let's say you can create new questions, can you keep doing that for years, because after each years, the current year of students will just made a compilation of known questions and sell it to the next year. New questions become known questions.

It makes it easier for students. Now, you may ask, why does it need to be easier? Simply because many professors know that students don't have time to study all of that. Life in university or college is really busy, and many students work part-time while attending classes. So having repeated questions each year based on lecture materials reduces the time required for studying.

It also reduces the chances of disputes. When you create new questions, different students will answer differently because they will have to answer based on their understanding. University students' wording is not always the best, so in cases where the answers to those questions were not regulated, it's hard to deny the answer of someone without disputes coming from them later on, which costs a lot of time and work.

-3

Well, I don't get what the alternative would be. As a student, I expect the exam material to reflect the notes, readings, and lectures. If the exam doesn't do this, then the notes, readings, and lectures are useless, from my vantage point.

I haven't started teaching yet, but it would be unfair of me to ask students details we've not covered in class or in the material, because everything I'll cover is worth covering under the time constraints.

I should note that the questions I would ask on an exam depend on the audience. If I'm teaching a masters level class on causal inference, I will only ask why, for example, the convex hull constraint is believed to be useful. I won't, for example, teach them about changing the constraints to allow for negative weights, allowing for intercepts, or taking away the adding to one constraint.

Why? Well because I honestly would expect very few masters students to even try to do well on a question like that. If I've only covered the classic method, then that's all they'll need to know about.

And even if it were a PHD course on synthetic controls, I would still at least talk about these different penalties and approaches to constraining the weights. I would demand they have knowledge of the basic setup, and then be able to talk richly about why changing these details might change our results. Why? Well, they're PHD students, and this hypothetical course would only be an elective, so if you're here in the class, I presume you want to know deeply about this method in econometrics.

You've already mentioned that your students are average (which is fine!!!). Why even bother to ask them things you know they won't do well on, if the median student in the class is average? Part of being a teacher is teaching to your audience, and putting things in a way they will understand, and testing them in a tough, but FAIR manner.

6
  • 6
    Surely you see the difference between what the OP was talking about ("the practice that some questions of a math exam are exactly from the course") and what you are talking about (" I expect the exam material to reflect the notes, readings, and lectures")? Jun 18, 2023 at 23:06
  • 2
    No, I don't see the difference (only in that I was much more broad in scope). If you're teaching quadratics to kids and you ask a question in class that's meant to be tough, and the question number for number reappears on the exam, this tells me who learned from their mistakes when they show their work. So, I concede I was much, much, much broader than OP, but the principle I'm talking about still applies. In other words, I'd want to hear why "the practice that some questions of a math exam are exactly from the course" is wrong/bad or suboptimal. Jun 18, 2023 at 23:17
  • 2
    (+1) I upvoted this (I believe I was the first to vote either way) because what you wrote is exactly what I was thinking when I read the OP's question. In fact, unless one specifically tells the students which proofs they will need to know, studying for such a test (at least for me) would be a lot more difficult than if I wasn't expected to reproduce proofs from class, since then I could simply focus on the techniques and concepts, and any proofs expected of me would have to be relatively straightforward or else hardly anyone would be able to come up with them in the time available. Jun 19, 2023 at 10:54
  • 1
    Also, for proofs given during the lectures, I think there is a huge difference between proofs the students are expected to know for the test and proofs the students did not know they were expected to know for the test. The former is quite common and reasonable, and the latter could easily wind up being questions no one gets correct. The only situation I can think of in which the latter applies in a nontrivial way are Ph.D. qualifying exams (U.S. math departments), and even for Ph.D. qualifying exams, students are given fairly detailed descriptions of what they are expected to know. Jun 19, 2023 at 11:00
  • 1
    I'm not sure I understand the first sentence ("Well, I don't get what the alternative would be"). The alternative is to ask questions that were not literally treated in class but test if the students understand and are able to use the concepts taught in class. Jun 21, 2023 at 5:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .