Guidelines for creating/designing an exam?

Are there (general) guidelines for creating an exam? I am a TA, and one of my tasks is it to design a written exam for the students. The students know that it will be based on one book, and on which chapters. I am using the questions after the chapters as guideline (and modify them). Course theme is (from nature science) derivation of formulae (which are also done in the book) and calculation of values by using formulae from the book.

Now my problem is: How many points should I award to which part? The students can choose between either using a hand-written cheat sheet or nothing. In the first case the formulas will not be awarded with points, in the latter they will. But I am not sure how much points I should give to a specific formula. 1 Point, if correct, 0, if not? 5, if completely correct, 2.5, if partly correct, 0, if not?
Thus, is there a general way of doing that, or do I have to experiment with it?

• It would be much helpful if you mention the course, is it a design course? Analysis course? History course? etc – The Guy Apr 19 '16 at 11:05
• Nature Science course, calculation of different material parameters and derivation of formulae – arc_lupus Apr 19 '16 at 11:07
• I'm not familiar with such a course, but for my engineering courses I like to do the following. I try to ask questions that has answers with multiple independent steps. I break the points based on the importance/difficulty of each step (avoiding carry-over steps). I try to design the course based on solved examples and end of chapter questions from other different books (other than the one the student use) taking into account the a similar level of difficulty. Also, try to ask colleagues for previous exams and to go over your exam. Remember to solve it and time yourself while doing that. – The Guy Apr 19 '16 at 11:15
• They had no homework in that course, and the course is a first-time course, i.e. nothing before. If I can solve the exam within 15-20 minutes, how much can I then assume for the students (90 minutes atm)? – arc_lupus Apr 19 '16 at 11:24
• I would say for a one hour exam, you would need to be able to solve it in 40 min. Remember you are accounting for the time needed for an average student to solve the exam. Not an A student. – The Guy Apr 19 '16 at 12:17

In my opinion the most important question is what should the exam measure? Depending on the institution, the type of exam, and the course this can vary a lot. Basically, exams are to learning what thermometers are to temperature. You want to have it as precise as possible, and not to build a barometer instead.

Once you know what you want to measure, check it once more: are these expectations realistic given the course and training given to the students? Are they enough to ensure that passing students will benefit from the follow-up courses? Here you will usually need some coordination with other courses.

I advise not to adapt the exam to the level of effort actually produced by the students, but to the expected level of effort. Since my students seem to produce much less effort than I expect, I usually give quite low grades (I always warn them of this fact). This may seem harsh, but in fact I think I am doing them a favor not to blind them with artificially boosted grades that let them believe they will do great with little effort in the following years, which is usually a lie. I have seen too many students stuck in third year with no hope to go forward, after hardly passing the first two years thanks to very generous grades. Of course, this point of view is hugely dependent on the environment.

An exam should be too short rather than too long given the allowed time. If it is too long, you will have to boost the grade to be fair, but that will probably give too high grades to students who master only a small portion of the course. If the exam is too short, you can still measure the level of mastery they achieved. The only exception is when you really want to measure their speed, but it is rarely relevant. My personal guideline is that I let my students three to four times as much time than I take to write the complete solution to the exam (I have been trained to take exams quickly all along my studies, your preferred ratio may differ but it is useful to establish a ratio in a way that ensures that time is not too much of an issue).

Another point to think about is whether you want to have your exam very close to the course itself (which is what student prefer and what gives the highest grades) or on the contrary to have parts which are significantly different, but rely strongly on the course material. This later choice is in order if you want to measure how much your students are able to use what they learned in another context. Be warned that results are often disappointing, and choose this option with care.

On the long run, try to adjust your exam depending not on the percentage of passing grades (unless you are forced into it, which often happens), but depending on the correlation between passing grades and future success of students. The information is often difficult to obtain, but if you realize that your students struggle to get a passing grade with you but have good success in follow-up courses nonetheless, you should be more gentle in your questions or grading. On the contrary, if many of your passing students struggle a lot with standard follow-up courses, you should probably be harsher.

In education, it is normally advisable to develop assessment prior to teaching. This allows the teacher to be sure that he or she is teaching to the assessment that they designed. When the assessment is made after teaching there is a risk that you will assess things you did not cover thoroughly or even assess things you never taught. Many scoff at this but it is common.

In terms of how much weight to give to various sections, this again is base on what you consider most important. Skills that are more important for students to show mastery should be worth more and lesser skills should be worth less. This is a modified application of bull's eye curriculum.

Lastly,all students should face the same conditions during the assessment. This means everyone has a cheat sheet and their formulas are marked or no one. If the conditions are different you cannot compare the results of one student to another, unless you are conducting an experiment and want to see if a cheat sheet makes a difference. If you can't compare results it is difficult to know who learned something and who didn't

• Overall your advice seems fine, but this sentence didn't make much sense to me: "When the assessment is made after teaching there is a risk that you will assess things you did not cover thoroughly or even assess things you never taught." I quite often adjusted test questions and content emphasis (from tests I gave when previously teaching the same course) to account to variations in what I ACTUALLY covered in the present class. – Dave L Renfro Apr 20 '16 at 14:22
• fair enough, but many teachers fail to make adjustments because the forget or get distract. – Darrin Thomas Apr 21 '16 at 0:13

Yes, there are general guidelines for creating an exam. As a TA, some of the guidelines aren't under your control, but here is how the instructor of record goes about things:

1. Determine the goals for the course. One goal might be "students should be able to apply these 10 equations appropriately in new situations."
2. Provide regular opportunities for students to practice the skills needed for this goal - practice applying equations in new situations. This can be done via homework or quizzes where students get feedback.
3. Design an exam that appropriately measures student ability to complete the goals. Each goal should have a number of questions that match the importance of that goal. The number of points for the full exam is usually determined at the start of the class.
4. Make each part of each question worth points of appropriate difficulty. If there are 100 points for the exam, and each question/equation requires roughly 10 minutes for a student to complete, then there should only be 4-5 questions for a 1-hour exam, and you would divide the points to match the difficulties of the questions. An easier equation problem might be worth 5 points, a harder problem might be broken into parts but be worth 25 points total.

THE MOST IMPORTANT THING is to "take" the exam yourself, and make sure you can easily award points to each appropriate answer or partial answer. Adjust the question parts so that the student answers are easy to grade (0 for poor or missing to partial credit to full credit). Ideally, you find a colleague to also "take" the exam to warn you about vague instructions or likely errors.

It surprises me that there are no old exams for you to look at. At my university, it is standard to make an old exam or a "sample" exam available to students (and new TAs) so everyone knows exactly what is expected. You may wish to look for graduate students who have taught the course before and ask to see an old exam.

A few tips based on my experience teaching undergraduate students for almost 20 years now:

• Make the first exam "easy." First-time teachers are usually the ones who make the most difficult exams. If your experience is the same as mine, you will be surprised at how many will fail your "easy" exam. Make the second exam "easier." Again you will be surprised at how many will fail it. Keep on making it "easier" until you find an acceptable percentage of passing students.

• Answer the exam before giving it and time yourself. Give the students three times the amount of time to answer it. So, for example, for a 1-hour exam, you should be able to answer it in 20 minutes (under the same conditions as the students').

• I don't think the exams should be tailored to match a given percentage of passing students. In certain situation (e.g. student have been appropriately selected in the first place and have worked appropriately) the passing percentage should be huge, and in others (e.g. students are not selected and everyone complains about the level of students the year above), it would be better to give harder exams. The student may vary from year to year, but the goal of a course should be more stable than that, and the exam should accurately reflect the mastering of the material. – Benoît Kloeckner Apr 20 '16 at 13:38