Is there such thing as too many references for one paper?

Question

I'm currently reviewing a paper that has an uncommonly large number of references compared to the average number for that particular venue (more than 60 versus around 30). This is a regular paper, not a survey one, and although there does not seem to be wrong references, it feels like some are not really necessary, and the fact that one particular author is cited more than 15 times makes me worry that this paper is used to inflate the number of citations of that author (the submission is anonymous).

Should I mention this in my review, and try to examine which submissions are "abusive", or just let it go?

Answer 1

It's really a judgment call… some people consider that more references is good, because it gives the reader a wider perspective into the issue, and some people consider it a bad practice (in old times because it wasted paper, but nowadays mainly because it obscures the more valuable information inside a long wall of text).

I have, as a reviewer, sometimes asked authors to cut down on the number of references, so it is certainly an acceptable practice. Here are some factors you may consider when making the decision:

• Does each individual reference bring something to the paper, i.e. is used to back up a fact, idea, or to give credit for a specific (and relevant) new idea that it introduced?

• Are references cited in block? I tend to consider it is very bad practice:

  A series of recent experiments have shown systematically that current human-powered aircrafts are not suitable for mass transportation [refs. 9–21]

  Each paper (or group of two or three papers, at the very most) should be used with regards to a specific point in the discussion.

• Somewhat disguised form of the earlier: are long lists of claims and references justified? Sometimes many citations are used as “examples”, where a few would suffice.

• Could some of the citations be replaced by one or two reviews on the topic?

• Are the work cited really the seminal work on each topic/idea/experiment, or are there also less “worthy” works cited?

Finally, it is true that there are some unethical practices that can lead to inflated number of citations. Excessive self-citation is certainly one, but there are others. For example, some authors cite very widely papers from all groups in their field, even when it is not really warranted, in an effort to help their chances at peer review: the idea is that the reviewer is less likely to be harsh to a manuscript that cites 5 of her own papers.

And in conclusion, if you believe that self-citation was the motivation, there is nothing wrong with reporting your doubts to the editor.

Answer 2

One¹ thing² that³ hasn't^1,4 been^5-7 mentioned⁸ yet¹ is^3-5 the⁹ effect^10,11 on^2,7,10 readability.¹² References¹³ are¹⁴ a¹⁵ necessity¹⁶ but¹⁷ loading¹⁸ up^19-21 a²² manuscript^23,24 with²⁵ too²⁶ many² will²⁷ often^3-5 make^5-7 it²⁸ more^9,29-31 difficult¹⁷ for³² me³³ to³⁴ follow^35,36 and^37-39 can⁴⁰ be^3-5 a^{13-15,19,21-23,25-27,29-33,35-37,39,40} deterrent.⁴¹ If^{2,3,5,8,13,21,34} a^{3,14,15,19,26} paper^2,7,18,28 turns^1,6,18 off^1,4,14,21,35 its^1,7,32 readers,^2,23 it's^{4,8,15,16,23,42} not^6,28 doing^{1,2,4,8,16,32} its^{2,3,5,7,11,13,17,19,23,29,31,37,41} job.⁴²

Answer 3

I can only see one way to resolve this and that is to critically assess if all references are necessary. Self-citation can be a problem but it may be perfectly fine in situations where the author is a leader in, for example, a small field. Excessive citations can also be a sign of the author not being able to weed out the critical papers from the "mass". In some cases, it may be tempting to provide all the literature found on a particular topic wher perhaps a review paper or relatively new paper summarizing past work could be referenced using the form "(e.g. author, yyyy)", indicating it is one of several possible references on the topic. I think it is perfectly fair to ask for such changes if the excessive referencing is clearly just excessive.

Answer 4

I do not think that there are inherent problems with citing some "extra" papers, if they are appropriately relevant. The purpose of citations is to help us form a network of research. If a paper is not cited, then it will not join that network. However, one does not need to cite every paper that an author has written in the past 5 years in order to maintain that network!

Rather than spend time examining each citation, perhaps you could ask the author why there are so many citations. If the author(s) has been inflating citations without due cause, then any academic worth their salt should promptly correct that indiscretion.

Answer 5

I wonder if the authors considered the question When you reference an article, is it always expected that you have actually read it?

I can imagine multiple likely reasons for the high amount of citations by one particular author:

• The over-referenced author is a member of the same group as your paper's authors (or even themselves) and they try to promote their own work. This may be adequate if the cited papers honestly led to the new one, but if they are just loosely related (if at all), the amount of citations should be severely reduced
• that author is one of the big-shots in the field, and the submitters either want to acknowledge that fact or they know that person is likely to also review the submission and would insist on these citations anyway - maybe this is actually already the reaction to such requests. And don't say this doesn't happen...

Unfortunately no matter the reason, to judge fairly you basically have to read at least the abstracts of said citations and check their relatedness. However, as F'x' answer suggests, if there are "obvious" signs of over-citation, e.g. block citations of more than, say, three publications, you shouldn't bother with that and rather directly state your concerns in your review.

Once the amount of references surpasses a sensible amount, the authors should maybe more directly justify the citation themselves instead of forcing each reviewer to figure that out on their own...

Answer 6

There should be a Goldilocks-range for this, and what this supposed to be depends on your field, topic (whether relatively new or not), and most importantly, your advisor and thesis committee.

(Image ref. PhDComics.)