Are open-book exams generally a superior way to test understanding on practical courses?

Question

I understand why it would be a bad idea to allow math students access to paper materials as it is necessary for students to know the basic math formulas by heart.

But what about advanced practical courses where it is important to "understand" rather than "know"? For example, knowing statistical formulas won't help if you haven't understood how to apply them. Knowing STL commands by heart is also useless for programming, unless you're actually good at programming. Intel's x86 developer manual won't help a complete newbie in Assembly programming.

Therefore the question is whether it is reasonable to believe that good courses should be focused on "understanding" the subject rather than "memorizing" it, meaning that the instructor should have no problems with his students using paper materials on the exam?

Answer 1

You might find this article on a computer science class valuable (1). The tl;dr is that sections allowed student-prep notes showed no overall improvement, but mostly because many students didn't bother to create good student-prep notes. Good notes = good grades.

A quick browse of a review/ meta-analysis article on the topic (2), indicates that open-note exams can decrease student anxiety, and that student-prepared notes tend to produce larger improvements than open-book exams.

As an instructor, I've found students often spend energy only where they feel it is needed. I would recommend providing sample exam questions (as a group activity in class?) that gives students a good idea of what will be required and what sorts of notes would be helpful. This will help them expend the energy in preparation instead of frantic in-exam page-flipping.

Thanks for wanting to trigger deep learning in students!

1. Duncan, D.G. (2007). Student performance shows slight improvement when open notes are used during information system exams. Journal of Information Technology Education, 6, 361–370.
2. Larwin, Karen H., Jennifer Gorman, and David A. Larwin. "Assessing the Impact of Testing Aids on Post-Secondary Student Performance: A Meta-Analytic Investigation." Educational Psychology Review 25.3 (2013): 429-443.

Answer 2

I believe there are two main counterarguments to this line of thinking:

• Examinations in which students are given unlimited access to material tends to lead to more challenging exams, since everyone in principle has access to all of the available material they can get their hands on.

• If unlimited resources are allowed, this potentially gives students who can access more materials an advantage over those who don't.

The second problem is the more serious, in my opinion, but both are major challenges that must be overcome.

Personally, I "split the difference": I allow students to bring in a limited amount of notes to help them. They can decide what they need to include or not, but it's their choice. Also, by placing the restriction, no one is inherently advantaged

Answer 3

When I don't want students spending a lot of time memorizing things, I simply provide supplemental information on the exam itself.

For example, imagine a course textbook lists 12 reasons why something might happen. Rather than expecting them to memorize all 12, I might list all 12 on the exam, with an application question such as:

The book lists these 12 causes; which of the 12 is most likely to lead to a catastrophic failure? Explain.

In the case of a statistics class where "it is important to understand rather than know," and, "knowing statistical formulas won't help if you haven't understood how to apply them," you could simply compile a list of formulae from the course textbook and put them on the last page of the exam.

By the way, I agree with Moriarty's comment: "The act of carefully constructing this summary sheet is usually much more useful than the notes on it." My approach doesn't have this benefit, but it does have a few advantages of its own. For one, a student is less likely to get a better score simply because of a better-prepared cheat sheet. For another, the student might focus on understanding the material at a higher level instead of transcribing lower-level details.

My general rule is: If something is hard for me to remember without looking it up, then I don't expect students to memorize it for an exam. I will supply it instead.

---

Here's another idea to consider. (I haven't used this one, but I had a professor who did, and I admit I liked his reasoning.)

One professor of mine had a custom of administering open book, closed notes exams. (In other words, you could use your book, but not a cheat sheet.) Why this way, instead of the other way around? Because, he said, "Most students won't keep their cheat sheets, but a lot of them will keep their textbooks." There were no restrictions on writing inside the margins or the back cover (in fact, that practice was encouraged). Sure enough, I still have a book from his course, and I just opened the front cover, and found my cheat sheet still intact, with plenty of complex equations and hyroglyphics in my own handwriting.

As more and more campus bookstores offer e-readers, this approach might become outdated. But I still think it's worth sharing; I always appreciated how this particlar professor was always looking years down the road, as opposed to just the end of the term.