6

There are different metrics like the h-index which try to measure the impact in research of a scientist.

Is there a metric which tries to measure the educational impact of a lecturer or a tutor?

  • 1
    The h-index is a bibliometric which doesn't make sense most lecturers (though one can count citations of textbooks etc). The analogous thing would seem to be something like the maximum h such that X has taught at least h classes of at least h students. Internally, universities do look at how many students someone teaches, but I don't think there are any actual teaching indices. – Kimball Aug 2 '16 at 12:16
  • 1
    Also, impact factor in the title (a measure for journals) aren't the same kind of thing as h-indices in the question (a measure for authors). – Kimball Aug 2 '16 at 12:17
6

TL;DR: yes there are metrics that evaluate standards of teaching, but there are many and they vary widely. For the UK, the Higher Education Statistics Agency (HESA) collects performance indicators that include teaching outcomes. Because of these differences, the impact of these metrics vary, too.

There are two crucial problems. The first is: how you define impact? A good teacher can impact a student by inspiring to do great things- inside or outside of academia. If students leave university, how do you know what they do is a function of the teaching they have received? And even if it was that straight-forward, how do translate macro-level successes onto individual teachers? The second is: how do you compare such measures across institutions, disciplines, countries?

Most measures that try to evaluate the quality of teaching tend to be on the macro level. For instance, university league tables approximate quality of teaching using student-staff-ratio, teaching satisfaction and course satisfaction. See for example the Times Higher Education Ranking for US universities or the UK Guardian league table 2016. The THE explains here:

the Teaching metric is measured by five performance indicators: a reputation survey, the ratio of staff to students, the ratio of doctorate students to undergraduate students, the number of doctorates awarded per academic staff, and institutional income

All of these are approximations and make some assumptions about what is affected by teaching. For example, the number of PhDs awarded per staff: is this a function of teaching? In the US, where PhDs usually encompass 1-2 years of classes, this makes some sense. In the UK, where PhDs are much shorter, this is already a problematic measure.

Internal feedback mechanisms

Many UK universities use internal systems: at the end of each course, every student receives a feedback form with questions about how well they though the course was run.

This differentiates lectures from seminars/classes/labs, differentiates Teaching Assistants (TAs) from lecturers (if applicable) and includes questions on how quick the feedback on homework was, how well-prepared classes were, whether or not the TA made the subject interesting, whether they felt engaged and so on.

In my institution, every lecturer/TA gets sent their evaluations after each term, in addition to the departmental average. This way you can gauge how well you are doing according to your own students, and how well you do in relation to your teaching peers. As with any questionnaire or survey, methods are imperfect (students that don't turn up don't get to take the questionnaire etc) but it serves as an approximation. How well you do does impact your chances of promotion (this is anecdotal, however- this will likely differ from place to place).

External measures of teaching

The UK government plans to implement the Teaching Excellence Framework (TEF), akin to the Research Excellence Framework (REF). See here the House of Commons report (Green paper):

Inside the report, page 5 reads:

There is no commonly agreed definition of what constitutes good teaching in higher education. In a diverse higher education environment, we heard that excellent teaching may look very different across different subjects and across different autonomous institutions

Instead, the government wants to introduce metrics that can approximate the quality of teaching. The more recent white paper reads:

We define teaching broadly - including the teaching itself, the learning environments in which it takes place, and the outcomes it delivers. Such things can be measured: students assess their satisfaction with their courses, retention rates are a good proxy for student engagement, contact hours can be measured, employers choose to sponsor some courses, or work with some institutions, because of the industry-relevance of their offerings, and employment rates can be measured

The TEF is not implemented yet and reactions have included many concerns about the usefulness of the proxies and the bureaucratic burden that collecting these measures will imply. The consequences of TEF include for example that universities can raise tuition fees if they do well.

Practical issues collecting teaching data: the example of retention

Since retention and drop-out rates (especially among minorities and working-class backgrounds) are very important in measuring teaching quality, the easiest way to collect that data is by recording student attendance. Both manual and electronic measures have their pitfalls: manually calling out 150 students in a lecture is time-consuming, electronic systems prone to failure or abuse.

Assuming that this is not a problem, the next question is: what to do with students that have missed classes? Recently, my institution has been obstructed to see to it that every student that misses two classes in a row, will have a personal meeting with their tutor. This has created huge pressures on tutors who have to see up to 60 students per week. A common problem of this system for example is that students missing classes due to illness are not recognised as such. Another question this raises is that perhaps vulnerable students drop out not due to lack of teaching engagement, but rather due to reasons that are beyond university control.

This example illustrates how difficult it is to 1. define measures of teaching and 2. to collect data on said measures. As with all measures, like the impact ratings of journals, they often serve a narrow purpose and should be used in addition to other pieces of information, and not replace them altogether.

Further difficulties arise from different types of universities (for example the difference between Universitaeten and Hochschulen in Germany, or Community & Liberal arts colleges in the US), and the differences between subject areas. A computer science class with many lab hours will require a different set of standards than the philosophy lecture accompanied by seminars. Many university rankings differentiate between subject areas, regions and age of university to counteract some of these issues.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.