44

Say I received a paper for review. Afterwards I discover that there are cited references that are not related to the discussed subject, or there are elements in the bibliography that are not referenced in the manuscript.

As a referee, is it ok to ask for the removal of such references?

I ask because I have never heard of someone suggesting the removal of references, but on the contrary, I have many times heard of researchers self citing or citing friends just to enhance their numbers.

  • 6
    It would also help if you can identify whether these spurious references are an attempt to boost citation count through self citation or if they're just plain mistakes. The first one hints to an ethical issue. If it's too blatant, my opinion is that it should be brought to the editor's attention. – Miguel Sep 28 '18 at 9:01
  • 3
    I wonder if a journal's production office has means to detect these "orphan references" automatically, and what happens when they find one. Anyone more expert than me with the journal toolchain knows? – Federico Poloni Sep 28 '18 at 9:53
  • @FedericoPoloni For having the reference in the bibliography without actually citing it you usually need to reference it with \nocite. Once the editor gets the .tex files he should be able to find the lines, which add the reference to the bibliography. – allo Sep 28 '18 at 12:01
  • 2
    @FedericoPoloni I've had copy editors point out bibliography entries that they wanted removed because they weren't mentioned in the text. Usually they were right, but in at least one case there was a citation in the text that the copy editor must have overlooked. – Andreas Blass Sep 29 '18 at 2:53
  • 1
    @Pancho: I reread you question and was embarrassed to discover that I had not addressed the issue of irrelevant references. I have updated my answer with a suggestion on how to handle this problem. – Carl Christian Oct 1 '18 at 11:24
68

Yes, the spurious references should be removed, as stated by @Solar Mike. There is a diplomatic way to accomplish this without making any assumptions or accusing anyone of anything. Write something along the lines of

The current bibliography contains a number of references which are not explicitly cited in the main text. They are: (insert complete list of numbers). The main text should either be extended to include a discussion of these references or the references should be deleted.

You compile the list by ticking of all explicitly cited references as you read through the main text. Any entries left are spurious.

A tool like LaTeX typically has a command which is used to print the entire bibliography. Spurious references are often generated by forgetting to disable this command when the final draft is compiled. If you can identify the tool, then you can insert a helpful statement to that effect, i.e.

As you are using LaTeX, please check if manuscript contains a \nocite{*} command.


EDIT: I just realized that my original text does not cover the case where the manuscript contains references which are cited in the main text, but appear to be irrelevant. A diplomatic approach can proceed along the lines of:

The author(s) should extend the discussion of the following references: (insert complete list of dubious references). Currently, it is not clear to the reader how these references relate to the subject under discussion.

This allows the authors to either explain why the references are relevant or remove them if they are irrelevant.

  • 19
    Some people actually do or import their references manually (i.e. those folks who use Word) and edits to the document might have made some citations from a previous citation list import superfluous. If this is the case, then this tactful catch-all is invaluable. – Pam Sep 28 '18 at 13:29
  • 17
    @Pam Word has support for citations/bibliography management; Granted a lot of people don't seem to know about that, but I figure an academic using it should. – Cubic Sep 28 '18 at 15:32
  • 5
    @cubic in my experience, citation management is insane pain in the behind even with the best practices and tools – aaaaa says reinstate Monica Sep 28 '18 at 17:12
  • 1
    Actually we're using Zotero or Mendelay or similar:) – Ajasja Sep 28 '18 at 21:31
  • 4
    @cubic you might be surprised at how many people do it the hard way ;) – Flyto Sep 29 '18 at 20:29
23

Let me suggest, more generally, that as a reviewer you have two tasks. One is to help the editor make a decision about acceptance. But the other is to help the author(s) improve the work. As such, you can suggest anything to the author that you believe will result in an improvement.

Authors don't have to take your advice, necessarily, though they are advised to consider it. Neither do editors have to take your advice.

Be professional and do your best job. In the current instance, as in many, it would be good to state to the author why you believe the references should be removed. Both the editor and the author can take that advice and act on it or not.

9

When it comes to suggesting changes, it is true that most reviewers will suggest where to add material. As the general purpose of these suggestions for changes is, however, to make the paper "better" (whatever that means in the concrete case), removing information can be fully compliant with that goal.

Now, you list two very different situations:

there are cited references that are not related to the discussed subject

These references do, by their mere presence, reduce the quality of the paper (if we assume a paper should be as concise as possible when it comes to providing information unrelated to the subject of the paper). You cannot truly tell whether such references were added with malicious intentions (for example, to increase someone's citation count) or simply by mistake, so you probably should not make any direct accusations, but a request to remove the respective references is well appropriate.

or there are elements in the bibliography that are not referenced in the manuscript.

This case, on the other hand, depends entirely on the guidelines of the venue you are reviewing for (provided that the references are indeed on topic for the paper):

  • If these guidelines require references to all bibliography items within the paper body, go ahead and suggest either their removal or adding a mention in the paper body.
  • If, on the other hand, the guidelines do not contain this requirement, keeping these references improves the paper, as it helps readers who browse bibliographies for related work in the subject area by pointing out additional resources.
  • 1
    Another possibility is that the references are relevant, but cited in the wrong place, and with insufficient explanation as to what the link is. In a recent review I pointed out that I didn't understand the relevance of two references. The authors responded by moving the references to another paragraph and adding a couple of sentences explaining what they had actually intended. – Anyon Sep 28 '18 at 23:30
8

I think that you should get erroneous references removed, the ones listed should be the ones used in the paper, no more, no less.

0

I myself have sometimes included a refernce in an article if it provided me with substantial background information related to the subject even though it didn’t apply to the precise subject or title content. Some articles are limited in length and the amount of introductory discussion must be kept short, but I would like to know where to look years later.

  • Nobody besides yourself will find this reference. I never read through the references. Maybe sometimes I check the ration of self-citations / overall references. – usr1234567 Oct 1 '18 at 6:09
  • @usr1234567 often when leaning a subject I will use a web tool that list papers cited by the papers I have already found. This includes when I don't hace access to the fall text. – Ian Oct 9 '18 at 8:42

protected by Alexandros Oct 1 '18 at 21:40

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.