5

As far as I understand, it is commonplace for editors to invite M reviewers when N < M would suffice. After N reports arrive, they take a decision and inform the authors. What happens to the N+1st, N+2nd, ... referee reports that arrive after the decision has been taken?

As a referee, I have only been told “thanks, we won’t need your report anymore” a very small number of times, and as an author I’ve never been told “look, there’s one more report for you”. So I wonder what happens in practice.

EDIT: I have in mind journals, not conferences. A conference is more driven by deadlines, probably.

12
  • 4
    as an author I’ve never been told “look, there’s one more report for you” – I have.
    – Wrzlprmft
    Jan 11, 2018 at 12:51
  • In practice, the journals I where I know the process have a deadline for the reviewers to hand in. If they have N reviews at that deadline, those are the N used and the reviewers who hand in later might not have any effect. If they have less than N, they would send out reminders and extend the deadline. If they have M>N reviews at the deadline, all of them will be used in the decision and given to the author.
    – skymningen
    Jan 11, 2018 at 12:54
  • @skymningen So the editor discards the surplus reports? Even when they suggest meaningful corrections that may be useful to the author? Jan 11, 2018 at 13:01
  • 2
    @FedericoPoloni - the journals I'm familiar with would want the request to contact the authors of a paper you reviewed to go through the editor anyway. At the least that means that all parties are aware of the communication and that there is no 'back door' that could impact the integrity of the review process.
    – Jon Custer
    Jan 11, 2018 at 14:20
  • 1
    I would really appreciate if we would get some first hand information from actual editors in the answers!
    – Dirk
    Jan 12, 2018 at 12:01

2 Answers 2

2

From my experience, things happen differently. Comments do highlight it correctly.

Initially, N reviews are solicited by the editor. If they disagree (or, I have to assume, some reviewers miss their deadlines), more reviews are solicited. I've been reviewer #5 few times.

Typical values for N are:

  • 2, with third as a tie-break – a typical grant application review scheme;
  • 3, with 4 or even 5 as a tie-break - seen more often with journal papers.
4
  • So you mean that typically N=M? There is no "referee overbooking"? So what is written here, for instance, is inaccurate? Jan 11, 2018 at 22:54
  • In my experience (chair of a small-ish track at a conference, IPC member few times, reviewer since, like, 2008): there might be an "overbooking" in a sense that M>N, but if more reviews are requested than needed and they all arrive, all are regarded. So no "overbooking" in the sense "your review is not valid". Jan 12, 2018 at 1:43
  • I think it's important to differentiate between a "request to review" and actual reviews. Sure, I'd ask more than N people to review. Then few decline, I'd still need to come up with at least N. If I get M>N it's not a problem. But all reviews that were submitted are considered. It's not like "oh, we got 10 reviews, so let's cherry-pick 4 out of there." Jan 12, 2018 at 1:44
  • I have heard some editors only "overbook" during "tough times" when they admit that many reviewers are either very busy or on holiday.
    – skymningen
    Jan 12, 2018 at 8:36
1

In my experience, typically the editor doesn't make a decision immediately after N reviews arrive. Instead (s)he waits for all the reviewers who've agreed to review to complete. This way the only chance of having surplus referee reports is if a reviewer agrees, does not submit by the review due date, editor makes a decision, and then the reviewer submits. This is rare, and gets even rarer the longer the editor waits after the due date.

If it does happen, and the original decision was 'revise', we write to the authors with "here's another review, please take that into account in your revision". If the original decision was 'accept' or 'reject', then the review is effectively wasted.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .