Is it professional for a professor to ask "surprise" questions on a test?

Question

To clarify, I am asking if it's professional to ask questions that while relevant to the subject/course, and are related to the topic, but have not been discussed in class, assigned as homework, reading, etc. and are also not related to any prerequisite class that the students should already know. I am also not asking about "gotcha" questions where it's a quick "know it or not" fact, but rather an entire procedure, proof, or concept of some sort.

For example in class, using a formula from the textbook to solve problems, but on the exam asking for a proof of the formula that has been used.

Another example - in a foreign language class, asking about a never-before-seen word that may be related to some other, studied words or that has a similar sound/meaning in the native language.

Do such questions "make students think out of the box", or are they more likely to discourage students when they struggle with them? While in the real world there may be many problems similar to this where the problem is completely new, is this something that should be taught in an unrelated class?

Does it matter if the test in question will be graded on a curve where even if a student doesn't answer the question correctly at all, they could still get a good grade?

P.S. - this question was prompted by the comments in this answer

Answer 1

Among the answers given thus far, the one which is closest to my thinking is that of xLeitix, and the side note about the context applies too.

Now,

Is it professional for a professor to ask “surprise” questions on a test?

It can be professional. As I said in a comment, it's unprofessional to ask such questions with the only purpose of failing as many students as possible.

Moreover, I don't like very much the adjective surprise in the title: a surprise is something unexpected, but if a professor clearly warns the students that at the exam they will find problems which have not been solved during the course, there is no surprise. So, in the following, I will talk about "new" problems.

Exams and tests have, as their main goal, that of assessing how well students master/understand a (small portion of a) certain subject. As others have well explained, new questions or problems can give a hint on how deep this understanding is.

But apart from the above stated main goal, exams and tests might also have secondary goals:

• An exam can be an occasion for learning new stuff. The well defined separation between learning and verification, which typically happens in a course, is something that drastically comes to an end when one starts working, even in academia. Learning and verification in everyday life are really interleaved, and many times learning has to be done along a stressful verification. So, a new problem during an exam can be an occasion to continue learning in a more "unprotected" way.
• An exam can be a hint, one of the many, that what has been taught during the lessons is not the whole story, and that beyond the lessons there is much more: new problems surely deliver this message.
• For a professor, an exam is an occasion to fish for good students to whom propose a thesis. Giving new problems can be a way to find students who are capable of independent thinking.

So, I tend to give new problems at the exams keeping in mind the above points.

To avoid being too general, let's make an example related to my experience. A few years ago I taught a course about sensors, transducers and signal conditioning circuits for graduate electronic engineers. The written part of the exam consisted in one problems about designing or analyzing a signal conditioning circuit or about evaluating the measurement uncertainty of certain transducer. Due to the vastness of the topic, the course could neither describe all kind of sensors and transducers, nor all possible signal conditioning circuits. So, I decided that every exam would have been made of a new problem, where "new" meant:

• A problem about a transducer not described in the course. Indeed the exam text contained a short description of this kind of transducer.
• A problem about the analysis and/or design of signal conditioning circuit not described in the course. The students, being electronic engineers, were expected to know how to analyze electronic circuits, even of moderate complexity. In more difficult cases, hints were provided.
• A problem about a known transducer applied in an unknown way.

Exams were open books and students could bring the solutions of all previous exams and all the class notes. After the written part, if successful, there was an oral examination which was more about class material.

What was the outcome of this kind of exam? The course was in general very well received by the students, even if the exam was considered hard: the percentage of success was around 30% (the pass grade is 60%). The major complaint was about the number of exercises solved during the classes, but this happens in all kind of courses. My answer to this complaint was that there were, indeed, time constraints that prevented us to solve more problems but, anyway, whatever the number of problems solved during the course, at the exam they would have found a new one (sometimes students ask for more solved problems in the hope that these will exhaust all possible cases).

From this and other experiences along 15 years, I think that students can withstand new problems at the exams as long as the motivations are well explained and, especially, as long as the course is worth of it.

Answer 2

I don't see a reasonable answer beyond "it depends." It depends on the question: some surprise questions are not actually so difficult and could reasonably be asked on a test with no special preparation, while others are very difficult indeed. It also depends on the students: you can demand more of experienced and talented students than you can of typical beginners.

There's certainly no rule that says you can only ask test questions that are similar to previous questions the students have seen. Sometimes asking unusual questions can be an excellent way to judge how well students have mastered the material. At the same time, test questions that are too unfamiliar or difficult can be unproductive. This is a balancing act that can be solved in many different ways, depending on the style of the person writing the test.

Answer 3

Every question on a test should be about the material in the course. Many times, however, the professor may be trying to teach a deeper concept than some of the students have learned. This is what creates a "surprise" question: the professor asks something that requires mastery of the material or insight into its deeper meaning, and the student has only learned material to a relatively shallow level.

For example, when I was a TA for a large undergraduate artificial intelligence class, the class taught two things simultaneously. The underlying concept threading through the whole class was how to think about data representation and problem decomposition. As part of teaching this, the students were also taught a number of standard AI algorithms. The tests then typically involved variant algorithms the students had never seen before. Weak students, who had learned the standard algorithms but not the underlying concept would often do badly and complain about the "surprise" questions, since they were being asked about an algorithm that they had never seen before. Strong students, who were learning the underlying concept, had no problems.

In general, then, encountering a "surprise" question means that the student is failing to learn the deeper concepts that the professor is trying to convey. Where the pedagogical problems lies, the professor or the students, is a completely different question...

Answer 4

I guess it depends on what you call a "surprise question". Usually, when you design tests, you don't want all questions to be the same difficulty. Rather, you would want to have a number of basic questions to find out who actually did not "get" the fundamental messages (and should hence fail), some intermediary questions which the majority of students will be able to do if they studied, and a small number of challenging problems, which are there to separate the excellent from the good students.

In my tests, "surprise" questions often form the "challenging" part of the test. I write them in the full expectation that only 10% to 20% of the class will be able to do them, but that is ok - not the entire class should have the best grade anyway. This way, me and others know after the course who the students that really understood the material were, and who just studied a lot.

Sidenote: I teach in a european country where it is usual to have a Gaussian distribution over the entire grading spectrum - it is not like in the US where having a "B" is often already considered a bad grade. Also, at least in undergrad courses, it is not uncommon that more students fail than have the best grade.

What makes "surprise" questions difficult for some students and attractive for many teachers is that they actually test understanding, transfer skills, and the ability to apply knowledge as opposed to mechanical memorisation of pre-learned procedures. This is easy to see in your "formula" example. A student that studied can apply the formula (he knows how it works, and how to apply it, and under which conditions), but only a student who really grasps the math behind it can do a proof that they had not covered before.

Answer 5

In my view, a fair examination question draws on any or all of the following:

1. Material discussed or presented during class contact time.
2. Material from any of the items on the course reading list.
3. Core material from any prerequisite courses.
4. Material that might properly be regarded as common knowledge for students at this stage in their education (basic mathematics, basic use of the English Language, etc.).
5. Knowledge that can reasonably be derived as a logical consequence of numbers 1.–4. Here, 'reasonable' is calibrated to the level of the course. Much more should be expected of graduate students who are essentially training to do point 5. for the rest of their professional lives.

In my experience, "surprise" questions usually come about because (a) students have not fulfilled their obligation to apprise themselves of the material in 1. and 2., or because (b) students are not sufficiently capable/comfortable with the subject matter to conduct the logical deductions in 5.

In either case, my view is that it is professional to ask questions that draw on all of 1.—5. In their capacity as educators, the main professional responsibilities of university teachers are to decide upon and deliver the appropriate material clearly, and to administer assessments capable of identifying students' success in mastering this material. A question that does not 'surprise' the majority of students can only test this mastery to a limited extent because it leaves little way to differentiate students who have truly mastered the subject from those who have merely done a good job of rote memorization. I would therefore view 'surprise questions' as an essential tool in professors' fulfillment of their professional responsibility as teachers.

Answer 6

My field is mathematics. I always tried to ask at least one question that looked quite different from anything that the students were certain to have seen before (though of course it relied on the relevant material), or that required them to combine several ideas that they might not previously have had to combine. On an exam in first-year calculus I’d have had at most a couple of questions of this type; on an exam in the more theoretical courses and in liberal arts mathematics courses I generally had quite a few such questions alongside the more routine ones, covering a range of difficulty. All questions, of course, required the students to write something, be it a proof, an explanation, or merely a routine calculation, and partial credit was always available.

I should point out that I was not grading to any pre-set scale. I have always preferred to construct the exam that I wanted and then interpret the results. Indeed, I refused to assign letter grades to individual exams, preferring to reserve that painful chore for the end of the course when I had as much data as I was going to get. Needless to say, I always explained all of this at the beginning of the course and again before the first exam. I also made it clear that I did not have the expectations to which most American students are used to being held: it generally worked out that the A students (apart from the rare curve-breaker) averaged 80-85% over the entire term — and I was not especially generous with A grades. A 50% average was generally a solid C.

Answer 7

All my real analysis tests (me being a student) were 50%+ completely new theorems to prove. It was to be expected that one would have to think outside the box to even pass the test. And I would say I learned an order of magnitude more in that class than I have in any with more standard tests. But that's a senior level course and being good at writing proofs from scratch was a skill that had been taught gradually over many lower level courses.

One could argue that being able to pass general tests with questions that require uncovered to vaguely-touched-on material or methods that involve combining techniques in ways unseen in class is the culmination of education. If that's in fact the case, it would make sense to introduce it early to cultivate the skill of being able synthesize new answers from covered material.

But it is important that it be reasonable for the student to know the prerequisite material to synthesize the answer. Don't ask a measure theory question on the first test in real analysis. Ask a question that requires use of the least upper bound property of the reals in a tricky way, for example.

Answer 8

I prefer a system where the homework problems are the most challenging. Also, I'm not in favor of a system where the homework is graded, because this makes it more difficult to choose good homework problems. The idea is that students learn best when they struggle a lot to solve difficult problems. One then has to accept that students may not have been able to do well on a particular problem, even if they are one of the best. Graded or not, homework should still be submitted and records should be kept about the student's performance.

The exam should serve only as a basic test that all the students who have seriously followed the course should easily pass. There is no way you can challenge the students in an exam that only lasts for a few hours as you can challenge them with homework that they would need to work on for several days.

The exam should be judged in combination with the homework. Each student's homework record (graded or not) should be taken into account when judging the exam. If it is found that the homework record is inconsistent with the exam performance, then the student should be invited to speak to the Prof. about the subject. It can be the case that the student was nervous and didn't see the solution to simple problems, such issues can be corrected in an oral examination especially if the student does not know that the meeting is in fact a secret oral examination.

It can also be the case that the student did not know much about the subject and just copied the homework assignments from other students.That will then become clear after speaking to the student, the student will then be given a failing grade for the subject.

Answer 9

Teachers have been asking questions that are not know to students for as long as teaching has been around. How else can a teacher make the student step out of their comfort zone where the answers have been clearly laid out and instead help them broaden their mental abilities. It is important that the questions is related to the subject being taught, but if I ask you a question about the really world cases rather than the hypothetical it will require some out of the box thinking. Yes it is professional. It is not if they are questions that the class has not prepared the student to answer.

Answer 10

There's no problem with Surprise questions as long as they're a small part of the marks and you tell students that there may be such questions.

Some of my professors would structure tests along the lines of 40% basics, 40% intermediate, 10% hard, 10% Surprise.

if you covered and understood all the basics you could reliably scrape a pass.

If you'd covered and understood all the material well you could get a good mark.

If you'd gone above and beyond and mastered the material on the course and had good general knowledge in the area you could get an excellent or perfect mark.

I liked the system as it meant in later years I could generally drag students I was tutoring through the exam based on the predictable stuff and you were also rewarded for independent study.

Answer 11

I don't think this has anything to do with being professional, but to what extent it is fair and/or desirable so ask such questions.

To me, the utility of unexpected questions that completely surprise students also depends on what kind of grading system they are used under, and this has not been addressed in the other answers. Under curve-grading/norm-referenced tests, one of the points of exams is to differentiate between students, so difficult surprise questions can be useful to e.g. test if students have gained a deeper understanding of a topic. Under such a system, it is reasonable (and to some extend desirable) that only a small proportion of students can answer some questions.

Under a criterion-referenced/goal oriented grading system, students are ideally supposed to know exactly what knowledge is needed to achieve a particular grade. Totally unexpected questions might be more problematic here under such grading criteria. However, what is an unexpected question is also subjective to some extent, and the learning criteria could also specifically mention fundamental understanding and the ability to apply the material to new situations. Even so, if a large proportion of students fail to understand or answer harder surprise questions this can partially be seen as a failure of the teacher/course (which not necessarily the case under curve grading), since this could indicate that the teachers have failed to convey either the required knowledge or the learning-goals needed to achive a particular grade (alternative explanations would be e.g. high goal standards or uninterested, lazy or weak students).

Answer 12

I see this problem spanning several levels of information:

On the bottom, we have course information that is memorized. Students should not be asked to reproduce definitions, facts, axioms, and other types of foundation information that have not already appeared in course material.

Next, we have "techniques." Especially in mathematics, we learn techniques for approaching hard problems. In other fields, techniques manifest as the methods we use to make inference, the types of reasoning that we use to interpret new situations. These are very general (like integration by parts, the epsilon over n trick from early analysis courses, or less rigid logic like the the broad historical notion that starvation corresponds with instability which makes revolution more likely) and can be combined in many interesting ways. Plenty of good, unfamiliar problems can be written based on familiar techniques. It can be very appropriate to ask students to develop a new technique to solve a problem on a test, as long as the intuitive leap is somewhat reasonable. This is a judgement call on the part of the professor that can reflect professionalism.

The ability of students to adapt new techniques generally depends on their grasp of broad overarching concepts. Development of entirely new concepts probably does not belong on tests because students are not likely to retain them very well under exam pressure. In general, exams are a method for gauging students' current knowledge/understanding of course material. I would say that it is more equitable and appropriate to introduce entirely new concepts on homework and in lecture, so that students have a greater opportunity to internalize them.

Answer 13

My answer is a bit biased in favour of students since I had a bad experience with such questions.

TL;DR; 'Surprise' questions are good and necessary in some cases but don't make them weight 50% of exam.

While I was a student we had a couple of professors who would start giving you surprise questions when you do relatively good in your examination (oral). Then they could find something you don't know and seriously decrease your overall result or even fail you.

There were some professors who have made surprise questions in written exams grading them as 40-50% of the test itself.

I personally hate this. In my case it had led to the situation when you try not to learn general course material but to anticipate surprise questions. Student have more than one course at the same time and sometimes there is not enough time (or interest) to have deep(out of program) understanding of all courses and you just want to pass the course with 75% grade or whatsoever.

It of course depends on the field of study. If it is something like theoretical physics it is necessary to have skills to think outside of the box.

I liked an approach of one of my school teachers. You could get 110-115% on the exam and the grade you receive is based on 100%. 90% you could get by regular questions and 20-25% by 'surprise' questions. So if you studied diligently you could have a good grade and if you have spent more time on the subject you could even cover minor issues with 'surprise' questions.

Answer 14

I guess it depends on the details, the country (different education culture) and subject (some of them are fact heavy, some of them are more problem solving focused).

I happened to come from a place where a student who can solve only problems in STEM field that explicitly showed in lecture or assigned as reading, considered to be rather mediocre (C level). I assume there are different educational approaches, too.

On the other hand, giving a question way out of blue can be off limit and just mean.

Answer 15

"New" material can be tested for on an exam if the students have access to or have been taught materials that are sufficient to make sense of the "new" material. This can apply to the following scenarios:

• Open-book tests. In an open-book test, it is fair to test students on material that they are capable of looking up on the spot and synthesizing with the knowledge they have gained in the course or were required to have in order to meet prerequisites. For example, in a test of French grammar where the students are provided access to a basic dictionary, it is fair to expect the students to look up a word and identify its grammatical and morphological categories despite the fact that the word never appeared in any of the lectures or required reading. Make it clear to your students which books/materials they should bring to the exam. Similarly, if your course is "Open Google", you can include a question that requires the student to figure out "Will a Hyperbarkonian Dehydrogenated Redonkulator output data that meets the following specification...?" as long as your course has taught the students how to read a machine data output specification and compare it to a requirements specification.
• Material that can be deduced from context. These are things where students can apply context clues in connection with course material to identify the likely meaning of the new material. An obvious, well-known example of this is reading comprehension through context clues, a technique commonly taught in lower grades. For example, consider the following question:

After Mary found out about the death of her father, she blargragathed for three hours until she fell asleep from exhaustion. Which of the following counseling interventions is most likely to assist her?...

If your reading comprehension skills are sufficient, you can probably figure out that blargragathing is a kind of negative emotional or behavioral reaction despite the fact that I made up the word on the spot for this answer. You can now apply the counseling interventions you learned in my course to answer the question.