Are standard metrics of academic output skewed by the relative popularity of a field?

Question

The number of citations seems to be a good unit of measurement for someone's success in a specific field, however, shouldn't the h-index also include the popularity of a given field? For example, I've seen papers in computer science being cited thousand of times while other papers relating astronomy only a couple-hundred times.

When taking into account the actual quality of the paper and the amount of work that was put into releasing the evidence, the astronomy paper would probably be measured higher (for example). However publications within less popular fields are cited less simply because they're not as popular as other fields.

If I choose a field that is not particularly popular, I risk at perhaps not achieving the same amount of success that I would if I had chosen a more popular field.

Are standard measures of academic output skewed by the relative popularity of the field?

Answer 1

The comments are already spot on, but let me elaborate a bit.

Comparing h-indices (or any other "hard" metric) is already dangerous in narrow fields and downright foolish if used for comparisons among different fields. This is not only because some fields are larger than others, but also because:

• Differences in publication standards. In applied CS we write lots of papers, in many natural sciences, much fewer papers get written per researcher and time period. Arguably, this is because many empirical fields require the setup and analysis of lengthy experiments, something that is not typically (but not never) done in CS.
• Difference in co-author ethics. Just check around here on this stack exchange, and you will see that standards for co-authorship are not at all uniform in all fields. Clearly, fields with more loose co-authorship standards also expect researchers to be part of more paper projects, hence leading to higher h-indices on average.
• Differences in citation standards. In some fields, papers traditionally have 10 or less citations. In others, multiple dozen references are considered an informal minimum, again leading to higher average h-indices.

(and this is even without going into how easy h-indices are to manipulate if you are willing to - keyword "citation rings")

Answer 2

Setting aside fields that are very small, with almost no one working in them, the size of the field matters quite a bit less than people tend to think. The reason is straightforward: larger fields have more citation donors, but also more citation targets. Suppose each paper cites 30 other references. In a field with a 1,000 papers, you would have 30,000 citations shared among 1,000 targets for an average of 30 citations received by each; in a field with 1,000,000 papers you would have 30,000,000 citations shared among 1,000,000 targets again for an average of 30 citations received by each.

In principle, the growth rate of the field does matter. If a field is growing rapidly, you have a large number of citation donors referencing a small number of target papers, leading to higher citations rates among these early entrants.

In practice, as mentioned in the other answers and comments, the most important factors are probably citation and authorship practices in the field, and the degree to which the field is adequately covered in the citation database you are using.