Is there any research/study/survey that looked at the pedagogical benefits of assessing students with closed-book exams for graduate-level courses (vs. open-book exams)?
I'm mostly interested in computer science and math education in the USA, if the answer is field- or country-dependent.
It depends on what you're trying to teach, and what you're trying to assess.
If your goal is to convey concepts, or to teach the things that everyone needs to be able to do without consulting references in order to be productive, closed book may be entirely appropriate.
If your goal is to test their ability to combine and apply the concepts, open book may be more appropriate.
(And yes, I too remember tests with "official" cheat sheets as a balance between these. Then again, I also remember one test whose official cheat sheet was essentially a set of mathematical jokes, because the test itself didn't require any of the rote-memorization material. Then again again, I also remember closed book tests where one of the tools I used was a set of mnemonics that would let me quickly scribble out my own cheat sheet for the formulas/simplifications I most needed -- I can still recite "quasineutrality, uniformity, equilibrium, low-level injection, steady state" but I'd have to hit the books to again remember how those assumptions were used.)
A Google search for "research on open book testing" gives many results. For example, there is a paper "Examining the testing effect with open- and closed-book tests" by Agarwal, Karpicke, Kang, Roediger and McDermott (http://dx.doi.org/10.1002/acp.1391) with abstract as follows:
Two experiments examined the testing effect with open-book tests, in which students view notes and textbooks while taking the test, and closed-book tests, in which students take the test without viewing notes or textbooks. Subjects studied prose passages and then restudied or took an open- or closed-book test. Taking either kind of test, with feedback, enhanced long-term retention relative to conditions in which subjects restudied material or took a test without feedback. Open-book testing led to better initial performance than closed-book testing, but this benefit did not persist and both types of testing produced equivalent retention on a delayed test. Subjects predicted they would recall more after repeated studying, even though testing enhanced long-term retention more than restudying. These experiments demonstrate that the testing effect occurs with both open- and closed-book tests, and that subjects fail to predict the effectiveness of testing relative to studying in enhancing later recall.
