How much experience should a teaching letter writer have with my teaching?

Question

When applying for academic jobs in the US, one is often asked for a recommendation letter from someone who can attest to your teaching abilities.

How much experience should the letter-writer have with your teaching? Should they have observed you teach multiple times? (In my present postdoc I will teach 3 courses per academic year, should I be inviting potential teaching-letter-writers to visit each course? Or perhaps a selection of undergraduate/graduate courses/seminar talks?)

Are the answers to the above different for postdocs/tenure-track jobs? Within tenure-track jobs, are they different for research-focused versus teaching-focused jobs?

---

I am personally in mathematics. I am assuming the answers will not be too field-dependent (please correct me if I am mistaken), so I left it out of the main body of the question.

Answer 1

In a situation where "teaching matters", there are two significantly different tangible assessment types: student evaluations, and peer (or supervisor) evaluations, not to mention the more extreme sort of commercial sites which have the systemic problem that the commenters self-select, typically with skewing to negative, even ranting reviews.

The aggregate response in university-organized end-of-term student reviews is useful, not necessarily as a gauge of whether you're a good/effective teacher, but whether the students are "happy". For that matter, successfully generating no "complaints" during a term is some kind of success. This does matter.

Peer-evaluation (occurring not at pre-arranged times, but unplanned, yes), or evaluation by your director of undergrad studies in your department, is the closest to what hiring committees might care about. In particular, if you are doing what your peers or "boss" think you should, it doesn't matter whether the students love you or not... as long as they're not actually unhappy.

(Indeed, our goal surely isn't so much to make students "happy" in a superficial way, as to achieve certain pedagogical goals.)

The attributes relevant to undergrad teaching, especially lower-division, are essentially unrelated to seminar talks and even to graduate-level teaching, since the objectives (and the frame of mind of the audience) are invariably very different.

Edit: a "teaching letter" that merely reports that the student evaluations were positive, or that there were no complaints, is positive, but cannot count as a "strong" teaching letter, I think. This is a special case of the weakness of recommendation letters of any sort that merely repeat facts not known first-hand to the writer. First-hand information is better, and, further, lack of a first-hand-information letter-writer is not a good thing...

Answer 2

How much experience should the letter-writer have with your teaching? Should they have observed you teach multiple times?

I don't think there is a "right" answer here, but there are a number of educated guesses that represent sensible approaches. I will list a few points that common sense suggests, as a starting point.

It makes sense that the person providing a letter should be familiar with your teaching at least "on paper." In other words this can be someone who has observed your teaching at least a couple of times. (I don't think it is critical that the observer has sat in on all the classes you teach, but maybe a couple sessions in one or more classes, to have some general idea of how you are doing. Chances are these were representative examples.) OR, it could be someone (like dept chair or your PI/postdoc advisor) who can be reasonably expected to have access to your end-of-semester course evaluations and has some decision-making capacity with regard to your teaching assignments.

should I be inviting potential teaching-letter-writers to visit each course? Or perhaps a selection of undergraduate/graduate courses/seminar talks?

I think both are fine ideas. Keep in mind the availability of those you are asking, so as not to overburden them with these requests. Sometimes once sit-in might be enough for an experienced faculty member to form a reasonably accurate judgment of your capacity for teaching. Kudos to you if you do not 'orchestrate' the specific class they sit in on, but invite them to drop by 'any time' on any class (except exam time). This is mostly for your own sake, so you can hold yourself to a higher standard overall across your classes. The confidence required to extend such an invitation may not go unnoticed as well.

Are the answers to the above different for postdocs/tenure-track jobs? Within tenure-track jobs, are they different for research-focused versus teaching-focused jobs?

Seems a bit like splitting hairs here...I believe any sensible approach should be sufficient across all of your purposes. The only caveat would be that common sense suggests that deeper familiarity with your teaching will allow the letter writer to write something more coherent about your teaching than the collection of generalities that may be the limit for someone only casually (e.g. indirectly) familiar with your teaching.

(Side note: There are more public though less formal records compared to course evaluations, like RateMyProfessors.com. Not the most authoritative resource on objective evaluation of teaching, but see how you are doing there, just in case.)

Finally, teaching simply doesn't matter as much when applying for research-focused jobs. If your core responsibility will be to drum up grant funding, worry about that in your application, and don't worry so much about your teaching record. This may not be universally the case, but should serve as a reasonable rule of thumb.

Good luck!