How can one learn from poor teaching evaluations?

Question

I teach at a large R1 state university, and I just received my teaching evaluations for Fall 2013. Usually I get excellent evaluations, but this time my evaluations for Calculus II were an unpleasant surprise. My numerical scores were mediocre, and representative student comments included: "Answers were obscure"; "can sometimes be cryptic when answering questions"; "didn't really answer questions".

These comments do not appear to be sour grapes, as the same students didn't complain heavily about the workload or grading of the exams. Moreover, I got excellent teaching evaluations when I taught Calc I a year ago, at the same university, with the same philosophy and style, with similar course policies, and with a comparable workload. Clearly, I did something wrong with regard to this course in particular.

I e-mailed both of my TAs, and only got encouraging comments ("I thought you did a good job"). I then e-mailed the class, and explained that my teaching evaluations were poorer than I expected, and asked students to offer criticism and suggestions for the benefit of future Calc II students. No responses.

So, apparently my teaching left room for improvement but I have no idea what to improve. This is quite uncomfortable -- is there anything useful I can do here?

Answer 1

Ben Norris's answer is excellent but I would add one additional point to it. When you are asking for constructive feedback from the students, you must do so in a way that students feel completely comfortable that their honesty is not going to come back to bite them.

At the end of every semester, I email all of my students a web-based survey with some open and some closed questions specifically so I can get their honest opinion. Again, the key is that all responses are anonymous. I believe if I asked them to email me (not anonymous) I would get nothing but praise, which does not help me improve at all. While I do still get many positive comments, there are usually some small gems in there which help me improve.

Answer 2

Here are two things you can do:

1) Since the comments you mention have to do with the way you answer questions in class, perhaps it is time to explain your approach.

Sometimes all you need to do is explain at the beginning of a course why you are doing certain things. For example, if you do not like giving full answers to questions so that your students still need to work out part of the answer (and it sounds like that is perhaps the case), explain on the first day of class why you think this approach is beneficial. Perhaps you noticed that your students were more engaged and did better on exams after you started this approach. Let your students know that! It will help them buy in to the strategy. Explaining potential peculiarities of your instructional approach is especially useful if you are teaching the second or third course in a sequence, and you did not teach the earlier courses (this is the case). The students are used to different styles. If they were used to an instructor telling them the complete answer all the time, then they will not like what you do unless they understand it. You probably had fewer objections when you taught calculus I, since you set the expectations for those students on how a calculus class would go.

2) Ask some of your colleagues to periodically sit in on your class.

This is a good way to catch negative behaviors that you might be unaware of. The other benefit is appearing open to constructive criticism about your teaching. Having colleagues sit in on your courses also helps separate "students did not like what I did" from "what I did was bad". Just because students did not like, does not mean that it is a poor method of instruction (see point #1 above).

Answer 3

There are several ways this can happen. scaaahu provides one good reason, that students found Calc II harder than Calc I and was not prepared for it. Another reason could be that some person or group of persons in the student group infect the others with a sentiment. I have seen this happen and it only takes one dominant person to get others on the train.

Your description of the evaluations and your digging into them, with no response form the students, should tell you that the problem primarily is not yours in terms of teaching etc. The only thing you may consider thinking about is how you introduced the class. Setting the tone at the beginning of the course (or earlier if that is possible in your system) and thereby preparing them for the course can be a powerful tool to reduce complaints. This is all about the expectations and if expectations are wrong, it may lead to discontent.

As I was writing this a good response from Ben Norris was posted so I can only agree with that reply and let my anwser add to his.

Answer 4

Not only have I taught engineering courses for 10+ years and had to have students (corporate students) fill out evals after every 2-4 day class but I also built/run the company's evaluation system. In my opinion the following are things that heavily influence evaluations:

1. The student's view of the topic. If students don't want to take your class because they hate math but yet have to fulfill a requirement then your evaluation will be lower - for sure. This is the #1 factor. You can easily see this if you add questions to your survey like "What is your interest in CLASS_FIELD?" (1-5) or "Why did you take this class?" (choices being part of major, liked topic, whatever, other).

2. Students want learning to be easy. If you made them do a lot of nonsense work for little payoff they will not be happy. I had a teacher make us write these essays once a week and the 15 essays were 10% of our grade. Just a ton of work and it mattered very little to our grade. "How would you rate the workload (I don't like that word but you get it) of the class taken?"

3. Be clear about your goals of the class. Make sure you discuss at the beginning what you will cover and a brief outline of chapters in a book, other materials covered, and if there will be class discussion questions not found in those. You do not have to tell them exactly what topics are on the test but there needs to be a happy medium between "Know Everything" and "Here are the exact topics". Your survey should have a question that says something like "Were the tests and assignments reflective our your expectations from the syllabus?"

4. As a teacher you need to make sure that your goals are aligned with the school's goals. Is your goals to have happy students after your class? Seems like the easier classes would rise to the top then or the classes that are more topical at least. The way to truly evaluate you as a teacher is to test their retention of the materials at 3-6 months. Not a flat out test, but do they still understand the concepts of the class? Even this can have a lot of noise because batches of students will fluctuate (but you could fix this with a pretest).

5. Culture and individualism. Nothing you can do in an anonymous survey to get around this. Basically there are certain cultures and groups that feel like a 3 out of 5 is really really good. While others may think that is horrible. You can label whatever but you cannot account for this noise in numbers. However you can figure out if this is the issue with blank essay boxes - at least one that is mandatory. If you really want a good mandatory feedback question (which is negative) "What about this class would you change?"

6. Knowledge of the instructor and comprehension level of topic. You are teaching Calculus I and II right? Moderate level of difficulty. So you probably get some brownie points with students if you know your stuff well AND more importantly you can explain the difficult points in an easy to understand way. I see instructors at my company get good scores because they are an expert (maybe the only expert) in a field. Some of these people can barely form a coherent thought but still good scores. But still that is the expectation of some students - they want the best/smartest teaching them. So... "What was your instructor's skill level on the classroom topics?" (1-5) "How well did your instructor explain classroom topics?" (1-5)

Now how do you make students fill out an eval. Well in my company (100 instructors) we tell them the eval is used to do attendance so they don't get credit without it. If your school cares they would do the same. If you want to know what range of questions get people to respond, I gave some hints but that is a different question.

Also we tend to call our evaluations the "happy forms". This is because generally the instructors act all happy before giving the online surveys out (they are fully anonymous and they generally follow Kirkpatrick I). I have witnessed instructors saying all kinds of positive things to their class and even some passing out treats during the eval/survey. Of course students will give the instructor better scores. Your variance from one class to another could have had just as much to do with your attitude and mood the 30 mins leading up to your survey than compared to the entire semester.

Answer 5

Don't wait for the end-of-semester evaluation. It does not give us a chance to improve the students' experience. And it usually causes the teachers a lot of remorse and confusion. Instead, incorporate a mid-term evaluation. Send online questionnaire to students and solicit their comments on aspects like i) if their expectations are met, ii) if the objectives are fulfilled in a regular base, iii) challenges they face, iv) and suggested improvements. Address their concerns and lay out your revision right after you have checked the results.

Use anonymous channels such as online questionnaires (Qualtrics or SurveyMonkey) or forum (TodaysMeet, which works like Twitter.) Compared to getting students' criticism from their e-mail, I think I will have better luck to talk a tiger into giving me its hide.

Answer 6

In my experience, the noise in course evaluations is around 1 point out of 5. My evidence for that is that I once taught the same course twice at the same time with the same book, the same syllabus, the same homework, and very similar exams. Not only was the evaluation rating on my teaching different by almost one point out of 5, but the ratings of how appropriate the book/homework/exam was were also different by around 1 point out of 5. I had another similar experience TAing two sections of the same course (where the quizzes and exams were set by the professor) and where the class got rated 6/7 in one and 5/7 in the other. Students' expected grades have a huge impact on the ratings. So although you should definitely pay attention to your student ratings, it's also very important to smooth out the noise.

Answer 7

"Answers were obscure"; "can sometimes be cryptic when answering questions"; "didn't really answer questions".

This makes it sound to me that students never really understood the core content of Calc II. It was a problem I personally struggled with and needed a tutor to solve. Often students will be able to complete homework and quizzes of Calc II content (especially tougher content of sequences and series), even though they don't fundamentally understand what is going on. Calc II is quite a course, as many students test out of Calc I and their first college math course is Calc II. With often a new way of thinking, and representations of problems that students have never seen, Calc II is extremely difficult. I'd suggest (especially with like Power, Maclaurin series, etc.) that you take extra time to explain to students at the most basic level what is going on and move forward. Relate it to real life scenarios if possible, and give a few examples of where such problems are used in real life. I think they're missing key connections which make understanding the course a lot easier.

While I don't think you're a bad teacher, I think there is a slight disconnect here between you and your students. Obviously your level of understanding at the content is much higher than theirs, and what you may think is an easy subject to understand, could be the complete opposite to your students. While your students probably understand how to complete many of the problems in Calc II, from what you've said I doubt they have a true understanding of the content.