Is there a known relation between course pass rate and student feedback?

Question

I analysed the relation between the feedback scores of my courses given by students against the average grades of the students in these courses. I found a moderate correlation (Pearson r) of 0.595. i.e. if the grades are better the students tend to give better evaluations of my teaching.

While I kind of expected something along these lines I then correlated the pass rate (= students with a positive final mark divided by the number of students) of the courses and mean student feedback and found a striking correlation of 0.898 (note that for Pearson r 0.0 would be no relation and 1.0 would be a perfect relation).

At the time I grade the exam I do not know the students’ feedback and at the time the students give the feedback they have not written their final exam yet but they have the results of the midterm exam and several marked assignments/papers (the combination of both usually being a pretty good predictor of the final exam and grade).

My analysis is based on a relatively low number of 10 courses (amounting to about 610 hours of class room presence).

Therefore my questions are:

1. Is there a known relation between pass rate and student feedback? E.g. a publication that analysis this on a larger scale? Or are my results just an odd outlier?
2. If this holds true would it not question the meaningfulness of feedback scores as making a course easier gives better feedback (tenure decisions etc)?

Answer 1

Yes, this has been rigorously probed in a 1998 study called DUET at Duke University. The study is described in chapters of Valen Johnson, Grade Inflation: A Crisis in College Education.

The correlations you describe had been observed for many decades, but they don't do a very good job of probing what's going on. The correlation could arise because teachers' effectiveness is correlated with students' performance, and therefore with grades; because students give higher ratings in return for lenient grading; or because students attribute success to their own efforts, and failure to the professor.

The DUET study got around these issues with a design that used a pretext to get students to fill out two different online evaluations of their teachers. (This was in the early days of the internet.) They gave evaluations both before and after seeing their final grades. The results showed a time-sequence consistent with a cause-and-effect relationship, with students tending to change their evaluations after seeing their grades.

There is other evidence that student evaluations lack validity, including a famous experiment called "Dr. Fox," in which a very energetic and persuasive lecturer gave a great lecture with total nonsense as the content.

Personally, when I'm on a tenure committee or evaluating an adjunct, the only thing I find useful about the evaluations is if the written comments show a pattern of something objective, e.g., lots of comments complaining that the instructor was often absent.