26

What is the purpose of the “Comments only to the editors” section in peer reviews? It seems lack of transparency to me. What couldn’t you possibly tell the authors?

6
  • 9
    In case it's not clear from the answers, the box is there just in case you happen to need it. You are correct in assuming things should go in the "to authors" sections by default. The majority of reviews will leave the "to editors" box empty.
    – R.M.
    Mar 26, 2023 at 18:31
  • @R.M. I knew that from the beginning, I was just asking what a legitimate case could look like.
    – user354948
    Mar 26, 2023 at 19:25
  • 5
    I am rather ashamed at how often I use this box to apologise for a review arriving well past its deadline!
    – Aant
    Mar 28, 2023 at 15:26
  • 1
    @Aant You shouldn’t accept if you are going to do that. We authors wait for months and months on end for an article that can be read in two hours.
    – user354948
    Mar 29, 2023 at 2:48
  • 1
    @Aant Oh, you had said “well past” its deadline. A week is a second in journal publishing in my field, not “well past”.
    – user354948
    Mar 30, 2023 at 15:35

5 Answers 5

77

A few examples:

  • "I happen to know one of the authors personally and I know that they didn't consent to their name on the manuscript."
  • "Are you sure this topic is within the scope of your special issue?"
  • "I can't understand the authors. The other reviewers seem able to, however. Send the paper to them, not me. I don't want to see this manuscript again."
  • "Did you notice that between revisions there is a new author?"
  • "I'm not an expert on part X of the manuscript, but I'm an expert on part Y, and ..."
  • "I previously reviewed this manuscript for ____ journal. I attach the review I wrote for that journal, since the authors have not addressed my comments."
  • "I have reviewed so many papers for this group of authors. They all apply the same method to slightly-different problems. Scientifically, there's nothing wrong with this particular manuscript, but it's salami slicing."
9
  • 11
    @user354948 yes, peer review is anonymous, so you can't tell the authors that you know them. The third bullet point doesn't mean you aren't qualified, just that you cannot understand what the authors are saying, which could happen if there's a language discrepancy. The authors obviously know if there's a new author, but the editor might not if they're not paying attention, and it's something they'd want to investigate.
    – Allure
    Mar 26, 2023 at 5:20
  • 7
    To add to the examples, for ACL conferences, the confidential field is for "any information that you want to share with the area chair and other reviewers assigned to this paper. For instance, a very strong (negative) opinion on the paper, which might offend the authors in some way, or something that would expose your identity to the authors"
    – justhalf
    Mar 26, 2023 at 12:17
  • 1
    I don't disagree, but I'd say some of these examples may vary with the individual. E.g., I would probably put scope, new author (if there were no major revisions), and salami slicing (without using the words "salami slicing") comments in my report for the author's eyes. Also, some journals request that you place explicit comments about recommendation for acceptance/rejection in a report just for the editors (see user71659's answer).
    – Kimball
    Mar 26, 2023 at 14:44
  • 5
    @user354948 "Did you notice that between revisions there is a new author?" can be easily interpreted as a misconduct allegation, so it's best not to tell the authors directly, especially since they could easily have already explained to the editor (e.g. in the cover letter) why there is an extra author. Telling the authors that you know them doesn't break anonymity per se, but it narrows the range, and there's no advantage gained so there's also no reason to tell them. (But you should tell the editor about the possible COI.)
    – Allure
    Mar 27, 2023 at 1:11
  • 9
    @kmm I once had a variant of that where the authors addressed my comments in a special way: "I previously reviewed this manuscript for journal X. The data in that version contained several outliers that cast doubts on the whole analysis. Those particular data points have magically disappeared from the current version while the rest of the manuscript stayed exactly the same."
    – TooTea
    Mar 28, 2023 at 6:38
38
  • You can convey information that would reveal your identity.

    E.g., you had already refereed a previous version of the paper for another journal and the authors ignored your feedback. Or: you know that the authors intentionally refuse to cite some existing work.

  • You can be more direct and blunt.

    E.g., if the main result of a paper is roughly at the same level of difficulty as a homework exercise one could assign to a grad student, I might state this bluntly in more or less these words in the private field, but I would phrase it in a more diplomatic way in the report. The reason for doing this is not so that we can have a laugh behind the author's back, but to avoid discouraging and hurting the authors.

    This applies especially if the authors are e.g. Ph.D. students, whom you don't want to discourage from pursuing research even if the paper should clearly be rejected.

    There is nothing nefarious about this: clearly different norms of respect/politeness apply when talking to vs. when talking about a person. If they did not apply, it would be impossible to have fully honest conversations about other people's research.

  • You can use it to give a concise informal summary of the report.

    "Strong paper. Clear accept." or "Borderline. Either way is fine with me." or "Not a great paper but probably publishable".

  • You can use it if you are uncertain about what to recommend.

    One can choose to write an ambiguous report which the editor can use to justify either decision, and then convey your personal subjective opinion (which you might be uncertain about) privately.

  • You can use it to convey information that is irrelevant to the author.

    E.g., that you might not be able or willing to referee further revisions.

  • You can use it to address potential conflicts of interest.

    E.g., if you ask the authors to cite a number of your papers, you might use this field to reassure the editor that this is a legitimate request and you're not just abusing your position as a referee to get more citations.

3
  • 2
    Ok, some legitimate cases here in case you were fearful of your identity being disclosed or something.
    – user354948
    Mar 26, 2023 at 19:28
  • 1
    I typically use it for 2, 3, and 4.
    – Dawn
    Mar 28, 2023 at 4:11
  • 1
    I am also more blunt about the other reviewers in my comments on a revision. If I see another reviewer make comments I vehemently oppose, I would say that in a "politic" way in the comments that other reviewer will see and then be much more blunt in my notes for the editor.
    – Dawn
    Mar 28, 2023 at 4:17
17

There's differing opinions about this due to transparency, but I have been told by at least two editors (one from a big name journal) that they prefer your suggested disposition of the paper (accept without changes, minor revision, reject, etc.) and your opinions and justification on that matter should go into the editor comments.

This is because the editor makes the final decision, which may differ from the reviewers.

For example, consider the situation where all the reviewers say in the author comments to accept because they focus on the technical aspects. However, the editor is unconvinced that the novelty or interest is sufficient for the journal and rejects the paper. This causes confusion and complaints about why the paper was rejected despite two good reviews.

The other direction occurs too: a reviewer says in the author comments, reject due to novelty, but the editor believes they are unduly harsh and, overrides and accepts the paper. This obviously leads to less author complaints, but is awkward, can lead to some ill will with the reviewer and may give the impression of cheapening the journal.

Some people think that this is unfair due to transparency, but you can also find guides saying you should do this.

6
  • I feel that your first example shouldn't ever happen. If the editor is going to reject even if reviewers unanimously think otherwise, then the editor should not waste everyone's time by sending it out for review in the first place. Mar 27, 2023 at 15:06
  • 2
    @EspeciallyLime The editor could be wrong or not absolutely up to date in all aspects of the field. For example, the problem seems not that significant to the editor, but a reviewer says that they were just discussing the same or related problem at a conference, then the article could be very timely. The editor may give the authors the benefit of the doubt and send it to review, but if none of the reviewers says anything about how novel it is, then their opinion holds. It also may not be a complete waste of time, as reviews can often be transferred to another journal.
    – user71659
    Mar 27, 2023 at 22:08
  • 1
    @EspeciallyLime Editors from top journals in my field tell me that they get far too many positive reviews -- they may still reject because they are not just looking for good papers, but for the best papers of the year. They don't just need positive reviews, they need borderline rave reviews.
    – Dawn
    Mar 28, 2023 at 4:22
  • @Dawn if you have very strict requirements for interest level, that can and should be checked at the quick opinion stage by requiring enthusiastic support then. Perhaps this is field-dependent to some extent; I am looking at it from a pure math perspective where a full review round will normally take at least 6 months, and maybe significantly longer. This sort of avoidable delay to resubmitting to a new journal is unacceptable. Mar 28, 2023 at 8:22
  • 1
    Enthusiastic support from whom? How do you propose it be evaluated other than by sending it out for review by experts in the field? Plus, the reviewers comments should be useful, not merely a delay.
    – Dawn
    Mar 28, 2023 at 13:11
13

Apart from what others have already said, you can also use the "comment to the editor" function if you are not 100% sure about your asssessment--you can use the comment to tell the editor so and why. When I did my very first peer review, I told the editor that this was my very first review, and although I did feel confident in my review, I was torn between suggesting "reject" and "major revision".

2
  • 3
    Example: "This manuscript draws on both field of study X and field of study Y. I have expertise only in X. Please ensure at least one other reviewer has expertise in Y." Mar 26, 2023 at 12:59
  • 2
    I did this once when the system didn't have a button called "major revision", but options like "borderline". I chose the one I thought would most probably lead to the editor asking for a major revision (and not outright rejecting the paper), then used the "comments" field to state exactly this.
    – Sabine
    Mar 26, 2023 at 13:02
4

Your report to the authors and your comments to the editor serve two different purposes and should address two different audiences. Your report to the authors should help them improve the paper. When writing these reports, regardless of my opinions of the paper, I maintain a positive tone. I point out problems, areas of concern, etc. and propose ways they could address them.

There are components that feed into my evaluation that are not helpful to include in reports. e.g., my accept/revise/reject recommendation (particularly if it does not agree with the editor's final decision). This is where comments to the editor can be useful.

My comments to the editor are intended to help them make then communicate their decision. In my comments, I clearly state my accept/revise/reject recommendation and summarize the major reasons for that decision. If I worry that my concerns cannot be addressed by revision or I'm on the fence about a decision, I say as much and explain why. This is never more than a paragraph or two tops, but provides the editor a high-level summary of the thinking behind my recommendation which might otherwise be less clear in my longer report to the authors and whether I am likely to eventually recommend acceptance if revisions are requested.

By revealed preference these comments must at times be useful because I have seen editors closely paraphrase them in their decision letters (particularly when providing guidelines for a successful revision). This may also address OP's concern with transparency. Editors are free to repeat information they receive in the comments if they feel it will be useful to the authors.

2
  • Transparency is never about what the obscured side freely decides or chooses to reveal, rather, it is about not having obscured sides.
    – user354948
    Mar 28, 2023 at 1:22
  • I agree. This is what I was taught. In particular, I would never be harshly or bluntly negative to the author, but I would be blunt to the editor. In addition, my reviews are often several pages long, so I highlight my top 3 concerns in the info for the editor.
    – Dawn
    Mar 28, 2023 at 4:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .