What is an "evidence based" teaching statement?

Question

(I'm a PhD student in mathematics, for context.)

What do people mean when they say that claims in a teaching statement should be "evidence based?"

For background, I have been watching my wife go through drafts of teaching statements and she has been getting pretty relentless criticism that it's not enough to say, "I design my courses to achieve X, Y, Z," she somehow needs to provide "evidence" that the course achieves these things. I used to think this meant I should really keep very exhaustive data of how my students were doing (i.e. spreadsheets with test scores), but I'm not sure that's realistic as a PhD in math (as opposed to a PhD student in education).

What can I do now, as an instructor, to facilitate writing an "evidence based" teaching statement in the future?

Answer 1

I think the verbage "evidence-based" is often a little misleading to a STEM researcher. When we hear "evidence", we think of proofs or scientific experiments with significant sample sizes and controlling of confounders. Clearly, such hard data is not usually available in your teaching statement. However, it's also not really what people are asking for when the want to see "evidence" - normally, plausible anecdotal data is entirely sufficient.

More generally, what people often mean when they ask for evidence is that the current text is simply not sufficiently convincing, often because it "tells" rather than "demonstrates". It's not so different to how we tell PhD applicants to write their Letters of Motivation - everybody can write that they love teaching (by itself this is a meaningless, empty statement), it's better to explain what you do that shows that you love teaching.

I can give an example from my own teaching statement for promotion to Associate Professor a while back. The initial version had a paragraph that read something like this:

I redesigned course XYZ and improved student success as well as student evaluations.

I received the fair criticism that I should be providing more evidence, because the current statement isn't particularly convincing. So my final version instead read something like this:

I redesigned course XYZ [explain changes]. After implementing these changes in 2017, the percentage of failing students dropped from an average of 20% to 13%, despite a technically more challenging exam design. Additionally, the course evaluation improved from an average of 3.2 to 4.4. This leads me to believe that my changes were effective, both in terms of student learning and student motivation.

This is clearly not hard proof that my course is now indeed better - maybe my exam wasn't actually harder, or maybe I just got a smarter class this year. But it's a whole lot more convincing than simply writing "I did well".

Answer 2

Like other academic disciplines, the literature on education research includes evidence-based results for various teaching methods. The "evidence" being a measurement of student outcomes (how well students achieved specific learning objectives) for various methods in comparison (e.g. active learning vs. lecture). One way to argue for an "evidence-based" course is to cite literature supporting your course design, teaching methods, assessments, etc.

Answer 3

I agree with your concern that sometimes "evidence based" is sometimes tossed out with less-than honorable intention. Just as on this website the use of "citation needed" varies between useful and snarky.

According to my memory and Wikipedia (proper citation needed), the term was in use in medicine before it was used in education. Ideally, it refers to policy and population-based medicine more than individual medicine, as patients have a nasty habit of getting a disease not fully studied.

In medicine, there is some push-back from those who think the role of intuition and common sense are now downplayed too much. See the following and all the follow-up:

[1] Smith, Gordon CS, and Jill P. Pell. "Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials." BMJ 327.7429 (2003): 1459-1461.

In mathematics education (I am a math professor) there is similar tension, especially at the university level. There are lots of trials, some with good controls, of larger classes, like calculus or Newtonian mechanics 101. The small number of students who take more specialized classes, and the variations in student populations, classroom architecture, for example, mean that there is never going to be solid evidence from a trial that closely matches a lot of courses.

There is, however, a lot of informed writing on more advanced education that one might call a case studies. Given limited data, what can we say? For example, I have read things like the following:

[2] Braun, Benjamin, et al. "What Does Active Learning Mean For Mathematicians?." Notices of the AMS 64.2 (2017).

One can also look for items that cite an article, as there may be rebuttals.

In writing a teaching statement, or a curriculum re-design document, why not take the time to review some of the literature that is relevant? If you enjoy teaching, you should enjoy some of the education literature. Notice I said some. I would say the same about mathematics literature. Find a few sources you find informative and be sure to balance those out with common sense. Not everyone reading a teaching statement will be impressed by citations to the literature.