15

It seems like the traditional exam model (two midterms at 30% and one final at 40% of the grade, closed book) is flawed. For instance:

  1. Students are not allowed to access resources, whereas in reality they would
  2. Students get only a few minutes per questions, whereas in reality they would get days
  3. Students are not allowed to collaborate, whereas in reality they would
  4. Students cram for the exam the day before, and forget what they learned after the course is over
  5. Students focus on the exam to the detriment of learning the material and understanding real-world applications
  6. It is difficult to claim that the exam score demonstrates mastery of the course material in contexts other than an exam

But in practice, is it possible to improve on these flaws, without introducing major increases in workload for the instructor? Is there any system of evaluating student performance in an undergraduate course which is an improvement over a traditional exam approach?

| improve this question | |
  • 4
    One argument against some of those points is that exams are meant to test knowledge, not ability (the difference being I have no idea what Henry III did, but I can easily look that up--I'm being tested on whether or not I can personally tell you anything about Henry III, not my ability to type Henry III into Google). – waiwai933 Apr 22 '14 at 20:00
  • 4
    This depends completely on the nature of the subject. Elementary Spanish is different from calculus. An upper-division history course is different from either of these. – user1482 Apr 22 '14 at 20:30
  • @waiwai933: I assume you mean "mastery, not knowledge". – JeffE Apr 22 '14 at 20:33
  • 4
    "Students cram for the exam the day before, and forget what they learned after the course is over" - That's right, but the next time they need the knowledge, the re-learning process will be much quicker than the initial learning. Such is the nature of all learning, not just for exams. – Ana Apr 22 '14 at 22:19
  • 1
    Collaboration poses an issue. What if there is only - student taking the course? Or students are only studying via distance? Actually the midterm exam is foreign to me. All subjects that I have enrolled in and also taught have had 2-3 assignments and 1 final exam. – Stephen Tierney Apr 23 '14 at 3:13
23

One alternative that's been gaining momentum in actual classroom use is standards-based grading (SBG): http://www.fwps.org/teaching/sbe/grading-system/

In SBG, the instructor establishes a list of milestones that students are expected to attain throughout the course. The students then can provide evidence of any sort -- within parameters set by the instructor -- that prove they have met the standard. Student evidence of attainment of a standard is marked on a scale usually from 0 to 4 (unacceptable, novice, progressing, acceptable, and mastery -- or something like this) and grades are assigned at the end of the semester based on how many of the milestones have been met at "Acceptable" level or above.

For example, in calculus, one standard might be "Take the derivative of a second-degree polynomial using the limit definition". A student might show that they know this by doing a problem on a standard timed test. But maybe they don't have it down as well as they should, and on the test their work is marked as a 2 out of 4 (progressing; maybe they got the definition right but did some of the resulting algebra wrong). This isn't the end of the story. Later in the course, the student can show evidence again that they've learned what they need to learn -- for example, they can schedule time in office hours to come work a problem or two to show you they've met the standard. Or maybe take a short quiz in class, or work a problem during unstructured group work time in class meetings, or whatever avenue the instructor allows.

The point of SBG is that we want to assess students based on what they know, and give them multiple ways to show that they know it. SBG can therefore be a superset of traditional timed testing -- students that do well in timed situations will do fine in SBG, but students who struggle with timed testing can have multiple chances to get their act together, and as long as they can prove they've mastered the material by the end of the course, that is what matters.

I don't personally practice SBG but I would like to. I'd suggest this blog post by my colleague Jon Hasenbank who is a big SBG proponent: http://profjonh.blogspot.com/2014/02/sbg-mia14.html

| improve this answer | |
  • 1
    +1 This is the only answer so far that details an alternative to traditional exams, as requested in the question. – Relaxed Apr 23 '14 at 10:27
  • 1
    +1. This is what drives me to learn as a student. I couldn't care less about my measured worth based on a 1-2 hour test. What I accomplish is really important to me, and exams have never been an accurate measure of what I have personally accomplished. I feel that working toward a goal (completion of a project, demonstration of principles in practice, etc) are worth much more than answering questions on an exam. While exams are an "objective way to test performance," so are personal projects/(work in general) which demonstrate the obtained knowledge. – Chris Cirefice Jun 25 '14 at 3:55
  • I think this answer needs some updates. ;) The last paragraph is severly outdated, after all. – Raphael Jan 12 '17 at 23:46
15

Although exams have the flaws you described, there are also many positive aspects you have not discussed:

  • Exams are actually an objective way to test performance, in a "hard to copy" environment. Although one may disagree with what is actually being tested, objectivity is usually not questioned
  • They are fast. It takes 3-4 hours to test hundreds of students
  • They are universally used and most of the students are accustomed to them.
  • It is an excellent way to prepare students to perform under stress, a skill which is extremely useful in any work environment. All works / skills have some sort-of-testing (e.g. presentations, interviews). Even getting your driver's license requires testing. So, learning to do well in exams is a crucial real-life skill.

The only complementary (but not good enough for actually completely replacing exams) tool I can think of, is project assignments. Especially in CS, projects are a very effective tool to prepare students for real-work assignments. But they are not a good enough method on their own without some sort of personal written examination. They allow too much collaboration, copying and usually when done in teams, the good students do most of the work while others slack. While a professor might take measures to minimize this, it is impossible to avoid it 100%.

So, I do not think there is a universally better way to test students' performance than written exams. One may complement it with projects, assignments, oral examinations but in most of the cases, completely abandoning it is probably a mistake.

| improve this answer | |
  • 1
    One example from QFT: "Question 1 (20%), apply this mildly obscure theorem to get a classical quantity in this situation. Question 2 (80%): express the solution to ex. 1 in terms of creation and destruction operators." You can say that it is objective (either you know how to apply that theorem or you don't), but you don't really get an objective feeling of the knowledge of the student. I guess it is mostly a matter of terminology. – Davidmh Apr 23 '14 at 8:13
  • 1
    Others are poorly worded. When I was asked "historical meaning and experimental evidence of Maxwell equations", I explained the most important experiments and landmarks, all the way to Einsten's tensor; it got a low grade because they were looking for "$\nabla \cdot \vec B = 0$ means there are no magnetic monopoles". Others were "the definition is correct, but you didn't give an example, as you should have implicitly understood from the question, half points". [But I am tending to rant here]. – Davidmh Apr 23 '14 at 8:18
  • 3
    @David - I once took a college course called Tests & Measurements, an entire semester devoted to the art and science of constructing fair tests that accurately measure student comprehension of course objectives. It was one of the best (and most useful) courses I've taken in my life, and it addressed pitfalls such as the one you mention here. The course was designed for K-12 educators, but I think a lot of profs could have benefited from the material in that textbook. Your initial comment is on the mark: as a measurement tool, not all exams are equally accurate, and some can be outright bad. – J.R. Apr 23 '14 at 9:32
  • 1
    This answer is mainly focused on defending the statu quo and does not seem to address the question. – Relaxed Apr 23 '14 at 10:24
  • 1
    @user11596 - It's worth pointing out that about 75% of this question is taking swipes at the status quo, so defending the status quo is not irrelevant, but pertinent to the matter as posed by the O.P. There's more to this question than the bottom-line question. – J.R. Apr 23 '14 at 13:13
13

Students are not allowed to access resources, whereas in reality they would

  • Not necessarily so. Instructors can allow "cheat sheets," or allow open book exams.

Students get only a few minutes per question, whereas in reality they would get days

  • Not necessarily true. My boss often asks me questions and expects me to give prompt answers. I don't always have the luxury of asking for a few days to research something. If we are in a high-stakes meeting with customers from out of town, my organization's effectiveness might well hinge on my ability to ask or answer intelligent questions on the fly.

Students are not allowed to collaborate, whereas in reality they would

  • True, but exams are designed to measure individual ability and performance, not someone's ability to contribute within a group, or accomplish some objective as a group.

Students cram for the exam the day before, and forget what they learned after the course is over; students also focus on the exam to the detriment of learning the material and understanding real-world applications

  • Perhaps so, but that's what we get when we test on the minutiae and the trivial, as opposed to testing on the synthesis of high-level concepts (more on that later).

It is difficult to claim that the exam score demonstrates mastery of the course material in contexts other than an exam

  • Maybe so, but it's not difficult to claim that, as a general rule, if students are given identical exams, students who score above the median probably understand course concepts better than those who scored below the median (with some possible exceptions, due to factors such as test anxiety, and perhaps even a little luck).

In short, you bring up some possible shortcomings with what you call "the traditional exam model," but you can address some of these simply by redefining your parameters. Instead of two closed-book exams that count toward 70% of the student's grade, give three open-book exams that count toward 50% of the student's grade. Make one of them a take-home exam, and you at least put a dent in the "few minutes per question" problem. By reducing the exam percentage from 70% to 50%, you have an extra 20% to play with, so assign a group project worth 20% of the grade, thereby addressing the collaboration problem you mention.

As for cramming and concentrating on the wrong things, that's what students will do if you structure an exam that requires them to commit a lot of petty knowledge to short-term memory. I try very hard to make my exam questions address higher-level concepts, rather than nitnoid facts. I ask them to explain these concepts, often by weighing in on hypothetical debates. (Sometimes these debates aren't even hypothetical; I'll find a online discussion thread where a debate is raging, then paste it into my exam and ask them to chime in). In other words, insofar as I can, I test on what I want them to remember five years from now. If I want them to solve a problem, but don't care if they've memorized a requisite formula (because they'll be able to look it up anyway), then I'll just include the formula in the exam. I often tell my students, "Anythihg you would need to memorize will be put in the exam itself. If it's something I have trouble remembering off the top of my head, I don't expect you to memorize it for the exam."

But in practice, is it possible to improve on these flaws, without introducing major increases in workload for the instructor?

Ah, now, there's the rub. If you followed my suggestions here, look what I've done! There are three exams to grade, not two. These exams don't have a lot of multiple choice questions, and then there's that new group project I mentioned (which needs to be drafted up, assigned, and graded).

It's hard to get something for nothing; most meaningful improvements are going to come with some cost.

| improve this answer | |
  • The first point is important: allow students to bring material but ask harder questions, in particular not only knowledge-recitation questions. It's hard to succeed in exams that require independent thought by "cramming". – Raphael Apr 23 '14 at 10:25
  • For the most part, this answer does not seem to address the question, instead focusing on unconvincing rationalizations. – Relaxed Apr 23 '14 at 10:26
  • @user11596 - I'll grant you that my answer here focuses more on the premise of the question than on the question itself. However, if the premise doesn't portray an entirely accurate story, I think there is some benefit in addressing that. – J.R. Apr 23 '14 at 13:06
2

In courses that focus more on real world practical ability, assignments (like creating a project, software, etc) are a suitable alternative. It can be done during more time, using literature and, if preferred, in a small group.

A student still needs to defend an assignment (demonstrate it working and answer enough questions to be sure the work has not been copied from somewhere).

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.