Where are all the citations going?

Question

I started noticing something that left me wondering a bit. My estimate is that the average paper (for example submitted to physical review journals, contains of the order of 50 citations). But if the average paper cites 50 other papers, then each paper should also receive 50 citations on average.

This is definitely not my track record. One could argue that cited papers are dominated by a few rare events (very important papers with thousands of citations), but by the very definition of an impact factor this cannot be the case, (unless one considers books, but I would assume they comprise max 20% of the citations): only top journals as nature reach an impact factor of this order, but these are far from the only papers cited. Many of them are other physical reviews, with an impact factor of order 3.

So that leaves me wondering, where are the missing citations going? Is it just a matter of delay, or is there a flaw in my logic?

Answer 1

The impact factor is not the right measure. The impact factor counts the average number of citations a paper receives in the two years after publication. But in most papers, most of the references given are for papers (substantially) older than two years, and consequently they don't count for the impact factor of the journals cited. That means that the average impact factor over all journals is not equal to the average number of citations.

Second, as others have pointed out, the distribution is heavily skewed. That's true for papers in general, but also for papers by the same researcher. To take just my own papers (see this google scholar page), there are a couple with over a thousand citations, but #10 on the list is already below 100 citations and there is quite a long tail of papers with far fewer citations than they have references themselves.

Answer 2

First of all, you cannot assume that citations follows a normal distribution. I'd wager that it is rather skewed, with many papers receiving few or no citations, and a few reaching several thousand. This, however, does not help you too much, when you look at the citations of your own work - which is most likely not in the latter category (so few papers are...)

For myself I find it useful to also take time into consideration, as citations accumulate. If I take my own papers, which are more than a few years old (excluding stuff like conference proceedings), I actually average around 50. But of course, if I include papers from last year as well, they drag the average down. Have you tried performing that exercise?

Answer 3

I suspect that this phenomenon has two main causes:

1. There is a delay between publishing a paper and having it cited. If you look again at those papers, you might be able to get a sense of the distribution of the time between when each cited publication was published, and when the citation occurred (i.e., when the present paper was published). It is highly likely that you will see that many of the papers cited have been written several decades before the citation. Since citations accrue over time, earlier papers have an advantage, and you will not get parity between the number of papers cited in a present paper, and the number of citations expected for that paper. Even in cases where this happens (e.g., a paper that cites 50 papers ends up with 50 citations itself), that may take a substantial amount of time to accrue.

2. As you say, there are some key papers in disciplines that end up being cited as background literature very frequently, and these papers accrue thousands of citations. The number of citations in a paper tends to vary between about 5-100, but the number of citations accrued by a paper operates on a logarithmic scale, where there are papers with citations that are orders of magnitude above the norm. (Noorden, Maher and Nuzzo 2004 find that the top 100 cited papers each have over 12,000 citations.) Statisticially speaking, this means that the distribution of citations received is much more positively skewed than the distribution of citations given, and so there is more "concentration" of the former in less papers.