6

I analysed the relation between the feedback scores of my courses given by students against the average grades of the students in these courses. I found a moderate correlation (Pearson r) of 0.595. i.e. if the grades are better the students tend to give better evaluations of my teaching.

While I kind of expected something along these lines I then correlated the pass rate (= students with a positive final mark divided by the number of students) of the courses and mean student feedback and found a striking correlation of 0.898 (note that for Pearson r 0.0 would be no relation and 1.0 would be a perfect relation).

At the time I grade the exam I do not know the students’ feedback and at the time the students give the feedback they have not written their final exam yet but they have the results of the midterm exam and several marked assignments/papers (the combination of both usually being a pretty good predictor of the final exam and grade).

My analysis is based on a relatively low number of 10 courses (amounting to about 610 hours of class room presence).

Therefore my questions are:

  1. Is there a known relation between pass rate and student feedback? E.g. a publication that analysis this on a larger scale? Or are my results just an odd outlier?
  2. If this holds true would it not question the meaningfulness of feedback scores as making a course easier gives better feedback (tenure decisions etc)?
  • Is there a correlation between the grade you awarded and the grades the students think they deserved? – Solar Mike Apr 22 at 15:47
  • @SolarMike Difficult to say as I do not know what the students think that they deserve ... Some of the failing ones did definitly agree at the exam review but others didnt – lordy Apr 22 at 15:56
  • I also have issues with student's comments - some good students never bother, while other students make more noise and don't always state the truth. – Solar Mike Apr 22 at 15:59
  • Seems like there wouldn't be enough variance in pass rates of similar-enough classes to do an analysis – Azor Ahai Apr 22 at 16:11
  • Seems pretty obvious. People doing well feel good. People who feel good are more likely to be positive. Teaching, grading isn't very special here. – Buffy Apr 22 at 16:30
7

Yes, this has been rigorously probed in a 1998 study called DUET at Duke University. The study is described in chapters of Valen Johnson, Grade Inflation: A Crisis in College Education.

The correlations you describe had been observed for many decades, but they don't do a very good job of probing what's going on. The correlation could arise because teachers' effectiveness is correlated with students' performance, and therefore with grades; because students give higher ratings in return for lenient grading; or because students attribute success to their own efforts, and failure to the professor.

The DUET study got around these issues with a design that used a pretext to get students to fill out two different online evaluations of their teachers. (This was in the early days of the internet.) They gave evaluations both before and after seeing their final grades. The results showed a time-sequence consistent with a cause-and-effect relationship, with students tending to change their evaluations after seeing their grades.

There is other evidence that student evaluations lack validity, including a famous experiment called "Dr. Fox," in which a very energetic and persuasive lecturer gave a great lecture with total nonsense as the content.

Personally, when I'm on a tenure committee or evaluating an adjunct, the only thing I find useful about the evaluations is if the written comments show a pattern of something objective, e.g., lots of comments complaining that the instructor was often absent.

  • 1998, wonder if anyone has noticed student's attitude and behavior patterns changing even only in the last 10 years let alone 20. – Solar Mike Apr 22 at 18:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.