I'm currently reviewing a paper that has an uncommonly large number of references compared to the average number for that particular venue (more than 60 versus around 30). This is a regular paper, not a survey one, and although there does not seem to be wrong references, it feels like some are not really necessary, and the fact that one particular author is cited more than 15 times makes me worry that this paper is used to inflate the number of citations of that author (the submission is anonymous).

Should I mention this in my review, and try to examine which submissions are "abusive", or just let it go?

  • 2
    Does the journal have a policy on number of references allowed? Many do (mostly print journals), and would force authors to cut down the number of references. – fileunderwater Oct 21 '13 at 11:13
  • As far as I know, there is no limit on the number of references, only on the number of pages, which is respected. – user102 Oct 21 '13 at 11:53
  • There is probably some discipline influence at work. In the social sciences, it is not uncommon to have around 50 references in a journal paper. – Shion Oct 21 '13 at 14:20
  • 3
    @shion: as I said in the question, the number of references is quite large w.r.t. the average number of references for papers in that venue – user102 Oct 21 '13 at 14:25
  • 7
    I once wrote a paper that contained all references to papers not referencing themselves. That was too many. – Pål GD Oct 21 '13 at 22:18
up vote 57 down vote accepted

It's really a judgment call… some people consider that more references is good, because it gives the reader a wider perspective into the issue, and some people consider it a bad practice (in old times because it wasted paper, but nowadays mainly because it obscures the more valuable information inside a long wall of text).

I have, as a reviewer, sometimes asked authors to cut down on the number of references, so it is certainly an acceptable practice. Here are some factors you may consider when making the decision:

  • Does each individual reference bring something to the paper, i.e. is used to back up a fact, idea, or to give credit for a specific (and relevant) new idea that it introduced?
  • Are references cited in block? I tend to consider it is very bad practice:

    A series of recent experiments have shown systematically that current human-powered aircrafts are not suitable for mass transportation [refs. 9–21]

    Each paper (or group of two or three papers, at the very most) should be used with regards to a specific point in the discussion.

  • Somewhat disguised form of the earlier: are long lists of claims and references justified? Sometimes many citations are used as “examples”, where a few would suffice.

  • Could some of the citations be replaced by one or two reviews on the topic?

  • Are the work cited really the seminal work on each topic/idea/experiment, or are there also less “worthy” works cited?

Finally, it is true that there are some unethical practices that can lead to inflated number of citations. Excessive self-citation is certainly one, but there are others. For example, some authors cite very widely papers from all groups in their field, even when it is not really warranted, in an effort to help their chances at peer review: the idea is that the reviewer is less likely to be harsh to a manuscript that cites 5 of her own papers.

And in conclusion, if you believe that self-citation was the motivation, there is nothing wrong with reporting your doubts to the editor.

  • 1
    I agree with @F'x. But how we can respond to the reviewers' recommendation to add their papers in the references (as I guess from those recommended papers comes back to special authors only)? – Hadi Sep 13 '16 at 8:35

One1 thing2 that3 hasn't1,4 been5-7 mentioned8 yet1 is3-5 the9 effect10,11 on2,7,10 readability.12 References13 are14 a15 necessity16 but17 loading18 up19-21 a22 manuscript23,24 with25 too26 many2 will27 often3-5 make5-7 it28 more9,29-31 difficult17 for32 me33 to34 follow35,36 and37-39 can40 be3-5 a13-15,19,21-23,25-27,29-33,35-37,39,40 deterrent.41 If2,3,5,8,13,21,34 a3,4,15,19,26 paper2,7,18,28 turns1,6,18 off1,4,14,21,35 its1,7,32 readers,2,23 its4,8,15,16,23,42 not6,28 doing1,2,4,8,16,32 its2,3,5,7,11,13,17,19,23,29,31,37,41 job.42

  • 10
    This answer may be too exaggerated. But, I still like it. – scaaahu Oct 22 '13 at 9:58
  • 8
    This answer is an example of the kind of paper that "turns off its reader", should this be downvoted then? – Vaibhav Garg Oct 28 '13 at 4:00
  • Very Nicely done! – Prof. Santa Claus Jan 24 at 2:54

I can only see one way to resolve this and that is to critically assess if all references are necessary. Self-citation can be a problem but it may be perfectly fine in situations where the author is a leader in, for example, a small field. Excessive citations can also be a sign of the author not being able to weed out the critical papers from the "mass". In some cases, it may be tempting to provide all the literature found on a particular topic wher perhaps a review paper or relatively new paper summarizing past work could be referenced using the form "(e.g. author, yyyy)", indicating it is one of several possible references on the topic. I think it is perfectly fair to ask for such changes if the excessive referencing is clearly just excessive.

I do not think that there are inherent problems with citing some "extra" papers, if they are appropriately relevant. The purpose of citations is to help us form a network of research. If a paper is not cited, then it will not join that network. However, one does not need to cite every paper that an author has written in the past 5 years in order to maintain that network!

Rather than spend time examining each citation, perhaps you could ask the author why there are so many citations. If the author(s) has been inflating citations without due cause, then any academic worth their salt should promptly correct that indiscretion.

I wonder if the authors considered the question When you reference an article, is it always expected that you have actually read it?

I can imagine multiple likely reasons for the high amount of citations by one particular author:

  • The over-referenced author is a member of the same group as your paper's authors (or even themselves) and they try to promote their own work. This may be adequate if the cited papers honestly led to the new one, but if they are just loosely related (if at all), the amount of citations should be severely reduced
  • that author is one of the big-shots in the field, and the submitters either want to acknowledge that fact or they know that person is likely to also review the submission and would insist on these citations anyway - maybe this is actually already the reaction to such requests. And don't say this doesn't happen...

Unfortunately no matter the reason, to judge fairly you basically have to read at least the abstracts of said citations and check their relatedness. However, as F'x' answer suggests, if there are "obvious" signs of over-citation, e.g. block citations of more than, say, three publications, you shouldn't bother with that and rather directly state your concerns in your review.

Once the amount of references surpasses a sensible amount, the authors should maybe more directly justify the citation themselves instead of forcing each reviewer to figure that out on their own...

There should be a Goldilocks-range for this, and what this supposed to be depends on your field, topic (whether relatively new or not), and most importantly, your advisor and thesis committee.

(Image ref. PhDComics.)

protected by Alexandros Jan 24 at 20:29

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?