5

The number of citations seems to be a good unit of measurement for someone's success in a specific field, however, shouldn't the h-index also include the popularity of a given field? For example, I've seen papers in computer science being cited thousand of times while other papers relating astronomy only a couple-hundred times.

When taking into account the actual quality of the paper and the amount of work that was put into releasing the evidence, the astronomy paper would probably be measured higher (for example). However publications within less popular fields are cited less simply because they're not as popular as other fields.

If I choose a field that is not particularly popular, I risk at perhaps not achieving the same amount of success that I would if I had chosen a more popular field.

Are standard measures of academic output skewed by the relative popularity of the field?

  • 2
    Usually you are competing with people in your own field, so the effect is not so great. But this is anyway yet another good point on why raw metrics are not to be followed blindly. – Davidmh Aug 12 '14 at 11:13
  • 5
    The popularity is even less important then the culture of citing which differs between fields. Comparing quality of papers / citation numbers / H-index or things like that between fields is imho useless. – The Almighty Bob Aug 12 '14 at 11:25
  • 4
    This looks like a call for discussion rather than an actual question, and is therefore off-topic. – user102 Aug 12 '14 at 12:06
  • Maybe I misunderstood the question, but it seems well-defined enough to me. – xLeitix Aug 12 '14 at 12:10
  • 12
    "The number of citations seems to be a good unit of measurement for someone's success" [citation needed]. – StrongBad Aug 12 '14 at 12:52
17

The comments are already spot on, but let me elaborate a bit.

Comparing h-indices (or any other "hard" metric) is already dangerous in narrow fields and downright foolish if used for comparisons among different fields. This is not only because some fields are larger than others, but also because:

  • Differences in publication standards. In applied CS we write lots of papers, in many natural sciences, much fewer papers get written per researcher and time period. Arguably, this is because many empirical fields require the setup and analysis of lengthy experiments, something that is not typically (but not never) done in CS.
  • Difference in co-author ethics. Just check around here on this stack exchange, and you will see that standards for co-authorship are not at all uniform in all fields. Clearly, fields with more loose co-authorship standards also expect researchers to be part of more paper projects, hence leading to higher h-indices on average.
  • Differences in citation standards. In some fields, papers traditionally have 10 or less citations. In others, multiple dozen references are considered an informal minimum, again leading to higher average h-indices.

(and this is even without going into how easy h-indices are to manipulate if you are willing to - keyword "citation rings")

2

Setting aside fields that are very small, with almost no one working in them, the size of the field matters quite a bit less than people tend to think. The reason is straightforward: larger fields have more citation donors, but also more citation targets. Suppose each paper cites 30 other references. In a field with a 1,000 papers, you would have 30,000 citations shared among 1,000 targets for an average of 30 citations received by each; in a field with 1,000,000 papers you would have 30,000,000 citations shared among 1,000,000 targets again for an average of 30 citations received by each.

In principle, the growth rate of the field does matter. If a field is growing rapidly, you have a large number of citation donors referencing a small number of target papers, leading to higher citations rates among these early entrants.

In practice, as mentioned in the other answers and comments, the most important factors are probably citation and authorship practices in the field, and the degree to which the field is adequately covered in the citation database you are using.

  • 1
    "Suppose each paper cites 30 other references." Right, that is definitely a key assumption. In fact the average number of citations varies wildly from field to field. I think you're right that expecting the number of citations to be directly proportional to the popularity of the field will not be accurate in some cases....which means one cannot easily "normalize" citation counts across fields by dividing by the size of the field. It seems to me that the variation is so complicated that one should not try to compare between fields at all. – Pete L. Clark Jan 17 '15 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.