19

Among the various indices for academic productivity/impact in the respective field, which is most accepted one?

You can see some productivity measures, used in academia, here and here

This question is inspired by the comment here and should not be confused with this

Clarification: The word accepted meant to be taken as accepted in the respective field of activity, for various requirements, say appointments, career advancements, selection for awards, invitation as an examiner, editor, reviewer etc. These measures might not have much impact on the general public, and that is not being asked.

Improve this question
1
  • 6
    The "eyeball norm". I am afraid.
    – Federico Poloni
    Sep 7, 2012 at 17:32

4 Answers 4

Reset to default
13

There is no accepted numerical measure for an individual's academic productivity. The available measures can sometimes be useful, but they all have serious weaknesses and many detractors. In particular, there is no widely accepted or safe choice: if you make any public use of a productivity measure, many people will react angrily, no matter which measure you choose (and their anger may well be justified).

Added in edit: In my experience, citation and publication counts are sometimes mentioned in letters of recommendation, but just as a crude numerical measure, rather than with any serious importance attached to them; most letters do not mention them. (I've never seen an h-index mentioned in a letter or job application, but perhaps it is more common in physics.) Hiring committee members occasionally impose minimal numerical standards, but just to rule out inappropriate cases ("we won't consider anyone for a tenure-track job unless they have at least two publications", say). In the departments I'm familiar with, nobody uses them to choose between serious candidates. Like Dan C says, they just aren't that useful: they add a small amount of information, with a lot of noise and even systematic bias.

Improve this answer
3
  • Adding a clarification. Accepted meant, accepted in job interview, promotions, etc in the respective fields than in the general public.
    – Noble P. Abraham
    Sep 8, 2012 at 1:00
  • 3
    There is no such measure in mathematics that is widely accepted for use in interviews, promotions, etc. (and I believe the same is true in other fields, although there are a lot of diverse academic fields out there, so who knows). Occasionally administrators impose these numerical measures on faculty, who generally complain that the results do not reflect genuine accomplishment. When I said "public use", I meant as opposed to using it secretly. For example, if a department announced that it was using the h-index to rank candidates, that would upset many people.
    – Anonymous Mathematician
    Sep 8, 2012 at 3:43
  • This is something about the number of departments/institutes that accept a particular ranking over some other ranking; leaving out the differences in opinion. Thus the 'accepted ranking' may be quantified.
    – Noble P. Abraham
    Sep 8, 2012 at 3:55
12

I agree with Anonymous Mathematician that likely any numerical measure will be insufficient. The best measure is that of esteem by your community. This will often be reflected in:

  • invitations to speak
  • inclusion in special issues of journals
  • election to society offices and/or
  • professional awards.

As much as administrations would find it convenient, scientific impact cannot be distilled to a few numbers.

Edit:

I think part of the issue is that Noble is asking for some metric that can be used by non-experts to judge a researcher in a field other than theirs. And essentially, what we (Suresh, AnonMath, JeffE, myself, etc.) are saying is: "No. You can't really get a good answer, without asking experts within the field of the person you want to evaluate." In some sense, this is a predictable power struggle. Outsiders want a way to do it on their own, and the insiders are saying "No, you can't do it right without us."

Improve this answer
5
  • Don't the esteem, some way or other related/tied to these measures?
    – Noble P. Abraham
    Sep 8, 2012 at 1:02
  • 2
    @NobleP.Abraham Only very loosely. For example, some groundbreaking results get cited dozens or hundreds of times. Other groundbreaking results are so hard that it's difficult for anyone else to say anything more on the subject. So the paper may get cited far less, even though it's a magnificent achievement.
    – Dan C
    Sep 8, 2012 at 3:23
  • Agreeing with you on drawbacks of such measures, but don't certain institute prefer some particular metric over other, when competent candidates apply for a job?
    – Noble P. Abraham
    Sep 8, 2012 at 4:03
  • 3
    PhD institution is important, but by far the most important is publications and letters of recommendation.
    – Dan C
    Sep 8, 2012 at 4:36
  • +1 for the edit. Yep, that's exactly what we're saying.
    – JeffE
    Sep 18, 2012 at 4:03
6

It is pretty clear to me that in fundamental mathematics, such a measure exists and is widely used: existence (and number) of papers in the five or ten most prestigious journals.

In France, publishing in one of this paper is very important for getting a professor position. We tend to pretend we judge people on the content of the papers, but the first thing we look at is for Annals of math, Inventiones and the like in publication lists.

The US system is less known to me, but it seems very important to publish in one of these journals to get tenure too, at least for some people: I have heard a well-established colleague saying that when he was refereeing for, e.g., Annals of Math (arguably the most prestigious of all), he asked himself whether the paper was worth giving the author a career for the rest of his or her life.

Improve this answer
5

I feel like pointing out that the very use of the word 'productivity' reflects a certain kind of bias towards "quantity". Ultimately, there are many different ways in which a researcher can make useful contributions to their field, and society at large. Maybe a key insight, a new way of thinking, a set of tools, or even a large group of students. Any particular measure of impact captures some subset of these, but there's typically no way to capture the entirety of a profile, and if you resort to numbers, the problem gets even worse.

Improve this answer
8
  • Thanks. Exactly, these measures are about quantifying a quality. While the metrics may not fully fulfil their purpose, they are being used for various purposes. I would try a simple example: consider two scientists applying for professorship, they have equal qualifications, in terms of number of publications, number of students graduated etc. The only difference they have is the number of citations to their papers. Their impact on society at large is somewhat a difficult quality to measure. So, these indices have to be utilized, for an unbiased award of professorship.
    – Noble P. Abraham
    Sep 9, 2012 at 1:01
  • 3
    I would argue that there is no unbiased way to evaluate scholarship, and so one must embrace subjectivity while being transparent about it.
    – Suresh
    Sep 9, 2012 at 2:36
  • Agree, there is no unbiased way
    – Noble P. Abraham
    Sep 9, 2012 at 2:38
  • 2
    I should also emphasize that all of the numerical metrics you list — number of publications, number of students, number of citations — are of secondary importance to the perceived quality, impact, and importance of the applicant's research record, as judged by experts in their field.
    – JeffE
    Sep 10, 2012 at 16:16
  • 1
    I don't think I'm saying anything that Dan, AnonMath, and Suresh haven't already said — None of the various indices are accepted as accurate measures of productivity, impact, or reputation.
    – JeffE
    Sep 11, 2012 at 21:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .