18

(I'm a PhD student in mathematics, for context.)

What do people mean when they say that claims in a teaching statement should be "evidence based?"

For background, I have been watching my wife go through drafts of teaching statements and she has been getting pretty relentless criticism that it's not enough to say, "I design my courses to achieve X, Y, Z," she somehow needs to provide "evidence" that the course achieves these things. I used to think this meant I should really keep very exhaustive data of how my students were doing (i.e. spreadsheets with test scores), but I'm not sure that's realistic as a PhD in math (as opposed to a PhD student in education).

What can I do now, as an instructor, to facilitate writing an "evidence based" teaching statement in the future?

  • 1
    Perhaps academia.se should require answers to be evidence-based (as skeptics.se does)... – Dan Romik Sep 24 at 23:36
  • 1
    So much work to do to appease the auditor or administrative side. The energy should be focused on revamping the course, instead of these menial clerk tasks. – kate Sep 25 at 7:01
  • It's one thing to say that you use evidence-based practices in your teaching (and back that up with citations to the literature) and quite a different thing to say that you've developed your own strategies and proved that they're successful. I'd expect in most cases that the hiring committee for an instructional position would be looking for the former rather than the latter. – Brian Borchers Sep 26 at 3:55
  • 1
    Most of the teaching statements for mathematics faculty positions that I've read in recent years (and I've served on half a dozen search committees in the last 10 years) describe the teaching approach of the candidate without any reference to research on mathematics education. These statements often state that the applicant's preferred teaching methodology is the traditional lecture, something that is hard to defend in the light of research in mathematics education. – Brian Borchers Sep 26 at 4:00
  • Is this for a job application or for a tenure file? – Elin Sep 26 at 7:21
26

I think the verbage "evidence-based" is often a little misleading to a STEM researcher. When we hear "evidence", we think of proofs or scientific experiments with significant sample sizes and controlling of confounders. Clearly, such hard data is not usually available in your teaching statement. However, it's also not really what people are asking for when the want to see "evidence" - normally, plausible anecdotal data is entirely sufficient.

More generally, what people often mean when they ask for evidence is that the current text is simply not sufficiently convincing, often because it "tells" rather than "demonstrates". It's not so different to how we tell PhD applicants to write their Letters of Motivation - everybody can write that they love teaching (by itself this is a meaningless, empty statement), it's better to explain what you do that shows that you love teaching.

I can give an example from my own teaching statement for promotion to Associate Professor a while back. The initial version had a paragraph that read something like this:

I redesigned course XYZ and improved student success as well as student evaluations.

I received the fair criticism that I should be providing more evidence, because the current statement isn't particularly convincing. So my final version instead read something like this:

I redesigned course XYZ [explain changes]. After implementing these changes in 2017, the percentage of failing students dropped from an average of 20% to 13%, despite a technically more challenging exam design. Additionally, the course evaluation improved from an average of 3.2 to 4.4. This leads me to believe that my changes were effective, both in terms of student learning and student motivation.

This is clearly not hard proof that my course is now indeed better - maybe my exam wasn't actually harder, or maybe I just got a smarter class this year. But it's a whole lot more convincing than simply writing "I did well".

| improve this answer | |
  • 3
    The funny thing is that this kind of non-scientific evidence is no better than the bald claim, and yet, people find it more convincing. Not intending to criticize you personally, but to truly convince me, either your claim has to be reproducible (i.e. if you teach like this, then you get that results) or it has to be backed by some other independent source (e.g. person X at my university said that it was helpful) that can in principle be checked. – user21820 Sep 26 at 2:48
9

Like other academic disciplines, the literature on education research includes evidence-based results for various teaching methods. The "evidence" being a measurement of student outcomes (how well students achieved specific learning objectives) for various methods in comparison (e.g. active learning vs. lecture). One way to argue for an "evidence-based" course is to cite literature supporting your course design, teaching methods, assessments, etc.

| improve this answer | |
9

I agree with your concern that sometimes "evidence based" is sometimes tossed out with less-than honorable intention. Just as on this website the use of "citation needed" varies between useful and snarky.

According to my memory and Wikipedia (proper citation needed), the term was in use in medicine before it was used in education. Ideally, it refers to policy and population-based medicine more than individual medicine, as patients have a nasty habit of getting a disease not fully studied.

In medicine, there is some push-back from those who think the role of intuition and common sense are now downplayed too much. See the following and all the follow-up:

[1] Smith, Gordon CS, and Jill P. Pell. "Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomized controlled trials." BMJ 327.7429 (2003): 1459-1461.

In mathematics education (I am a math professor) there is similar tension, especially at the university level. There are lots of trials, some with good controls, of larger classes, like calculus or Newtonian mechanics 101. The small number of students who take more specialized classes, and the variations in student populations, classroom architecture, for example, mean that there is never going to be solid evidence from a trial that closely matches a lot of courses.

There is, however, a lot of informed writing on more advanced education that one might call a case studies. Given limited data, what can we say? For example, I have read things like the following:

[2] Braun, Benjamin, et al. "What Does Active Learning Mean For Mathematicians?." Notices of the AMS 64.2 (2017).

One can also look for items that cite an article, as there may be rebuttals.

In writing a teaching statement, or a curriculum re-design document, why not take the time to review some of the literature that is relevant? If you enjoy teaching, you should enjoy some of the education literature. Notice I said some. I would say the same about mathematics literature. Find a few sources you find informative and be sure to balance those out with common sense. Not everyone reading a teaching statement will be impressed by citations to the literature.

| improve this answer | |
  • 1
    That first article was nice. I work with some former pilots and skydiving instructors. They totally agree that most parachute-related "evidenc"e is purely anecdotal :) – Mad Physicist Sep 25 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.