We want to compare the scientific outputs of researchers across various fields (such as theoretical physics versus organic chemistry) quantitatively. Is the following formula appropriate?

Let us consider researcher A in theoretical physics and researcher B in organic chemistry. First we obtain MIF (mean impact factor) for each researcher’s field.
Then, let us denote sum totals of the impact factors of each researcher’s papers published during a year to be S(A) and S(B). Now, we can divide S(A) and S(B) by MIFs of each field. The ratio of the values obtained for each researcher can provide us with a relatively appropriate ground for comparing the outputs of these two researchers, who belong to essentially different fields of research.

  • What did you understand from this question? academia.stackexchange.com/q/107506/72855 – Solar Mike Apr 4 '18 at 11:18
  • 1
    What is your MIF ? Where do you get the data? What is the error? Do you have access to all the data? Or what % of data is available? There are so many questions... – Solar Mike Apr 4 '18 at 11:19
  • 2
    I voted to close as "unclear what you are asking", mainly because 'appropriate' is a very vague measure. As usual, Stack Exchange does not display close reasons correctly. (Please upvote meta.stackexchange.com/questions/54917/… to change this.) – Federico Poloni Apr 4 '18 at 12:24
  • 2
    I am not sure why this question is closed - yes, it is asking something very broad, but it is very easily answerable as "you don't want to do that, here's a better way"... – Andrew Apr 4 '18 at 13:53
  • @ Andrew I think so. I am interested in this question and will follow comments and answers. – Ali Bagheri Apr 4 '18 at 14:22

Normalising across fields is hard, but it's worth attempting. I would recommend reading this PLOS Biology paper as a good introduction.

This proposed measure is not appropriate because papers do not have an impact factor. Journals do, but there is a very wide distribution of papers within a journal - an impact factor can, in some cases, jump an order of magnitude because of a single paper.

"Mean impact factors" also take no account of publication volume. If you have a journal with an IF of 2 and one with an IF of 3, it's tempting to say that the mean is 2.5. But what if the first journal publishes ten times as many papers? It gets complicated quite fast.

The approach of normalising against the field/year makes sense, but ideally what you want to do is compare the actual citations for the researcher's papers in year X against the actual citations for the field as a whole in year X. The hard part here is defining the "field"; most approaches do what you planned to do here, treat all journals in a field as equivalent to the field as a whole. This is the basis of the source-normalized impact per paper (SNIP) and mean [log-]normalized citation score (M[L]NCS) measures; a more complex approach involving defining the field by citation networks is the basis for the relative citation ratio (RCR). All the usual caveats about counting citations apply, and numbers should be assumed to have fairly wide error bars for single-person data, but the results will make a lot more sense than one purely based on journal-level averages.

These approaches are a bit more labour intensive, but should be practical if you have access to a tool like Web of Science or Scopus.

This will effectively normalise for age of the paper and for the different citation practices in different fields. What it won't normalise for is fractional authorship - these measures can give somewhat confusing results if you're comparing someone who's routinely fifteenth author on large collaborative papers with someone who works in fields where single-authored papers are normal - or with someone who's usually the lead author on the large collaborations. Likewise, it will give misleading results if either person works in a field with substantial non-journal activity (eg the humanities).

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.