I am a visiting faculty, professor in B-schools. How does one go about in designing or compiling an open book exam, ensuring that it's not tough and neither too easy. At the same time someone who has not read the book will not be able to answer the question. What at the pointers or measures that should be taken care? Isn't designing an open book exam difficult compared to traditional exam?
When designing an assessment, it is helpful to look at something like Bloom's Taxonomy. Since you mention B-school, I assume you are assessing a management or finance module. I always thought finance modules in particular should be open-book because of the extensive use of formulas and, as pointed out by F'x and others, simply testing if the student can remember some information is not really the most interesting (for teachers or students).
Sticking with Bloom's Taxonomy, remembering is the lowest form of assessment. If the students do not have to worry about getting the formulas right, then you can start something more complex. For example, have them make recommendations or create a plan for whatever the subject is. For example, if you are assessing leadership, give them a scenario with enough background knowledge and ask them to recommend (with justification of course) a particular leadership style for the situation. Remembering the standard leadership styles is not what is most important. What is important is that they can see a situation, assess the more important elements of that situation with respect to leadership, evaluate the pros and cons of each option, and decide which would be the best of all the choices to be used, which may include creating their own leadership style.
Basically, for open book tests, move further up towards evaluation and creating and away from rote memorization.
Open book exams are actually very easy to design (at least I think so, and I favor them in my own teaching): just design it the same way you would design regular exercises. The goal is to evaluate the students’ understanding and skills, i.e. how they analyze and solve problems, rather than the facts they have memorized. To do so, I usually design exams as problems rather than series of trivia questions.
Open book questions should be able to evaluate student understanding. Should be able to test the ability of the student to think and come up with innovative solutions. It cannot be completely open ended. there should be a sufficient number of constraints within the question to enable convergence and a positive solution. It is more difficult to set a question paper for an open book exam than a closed book one.