Editor invites many reviewers simultaneously

Question

I have been reviewing for a particular prestigious journal in my field for some time. I noticed that the editor invites a significant number of reviewers (4-7) simultaneously. In most cases, not all reviewers submit their reviews in time, but in one particular case, 5 reviewers submitted a review, and the paper was rejected.

While I understand that it is common to invite a number reviewers because many end up declining or not submitting a review, I feel like it is unethical to do so simultaneously, and is a situation similar to an author submitting to multiple journals, as it "wastes" reviewer-hours. In the particular 5-reviewer case, the paper did not need 5 reviewers to be rejected.

Is this kind of practice common and/or is it considered ethical in general?

Answer 1

It's quite common to invite multiple reviewers simultaneously, though 4-7 is on the high side in my experience (yet sometimes, even 4-7 is not enough!). The numbers quoted in the details below are for physics, and might differ for other fields.

Regarding whether or not it's ethical, it really depends on tradeoffs. The tradeoff for not inviting multiple reviewers at the same time is that your paper will take longer to handle. It's not just submitting reviews on time. Not every reviewer invited will agree to review it! My personal guideline when I was an editor was that, each time I invited reviewers (3-5 at a time), I would budget one month till I receive reviews. That itself is an optimistic budget - 6 weeks is closer to the average, since there's a long tail in the distribution.

The alternative to inviting multiple reviewers at the same time is to invite one at a time, and wait till you hear back from the reviewer if (s)he is willing to review. This leads to the following problems:

1. How long are you willing to wait to hear back from the reviewer? Let's say a week (which was also my personal guideline). Since you might have to invite 6+ reviewers before someone actually agrees to do it, you could easily be waiting 6 weeks before the paper starts being reviewed. If the reviewer takes one month to complete, that's at least 2.5 months before you can make a decision, not including any administrative time (i.e. you and the editor-in-chief both make decisions immediately).
2. What if your journal requires two reviewers? You can invite two at a time, but there's no guarantee that both of them will agree to review. Maybe one will, the other won't. You could end up with one review submitted with the next one due in another month.
3. What if the reviewer you invited writes a poor review (happens more often than one might think)? Sure you can blacklist the reviewer, but you still need to make a decision for the paper. If you invite more reviewers, again, that's another 1+ month till you receive the reviews.
4. Nightmare scenario: what if your reviewer agrees to review but then does not actually submit a review, in spite of multiple chasers? If we take the worst case, you've already waited 2.5 months and then discover that you have to start from scratch, which can lead to another 2.5 months before you can make a decision. Ouch!

In the end it comes down to how long authors are willing to wait for their papers to be reviewed. If authors are willing to wait 6 months, then yes, editors can invite one reviewer at a time and waste fewer reviewer-hours. On the other hand it's common for authors to start worrying if they don't hear back in two months, in which case the editor really cannot invite one reviewer at a time.

Answer 2

Editors are creatures of habit, just like the rest of us.

If they’ve been on the job a while, they generally have a good idea of how many invitations they need to send to get the desired number or reviewers. Sometimes, though, they end up with more than they expect. It happens. But I think the occasional example of this happening is better than inviting only a few reviewers at a time, and waiting for responses before continuing to the next group. That lengthens the time needed for review and slows down the cycle.

Answer 3

I've seen this in cases where referees were slow to respond, did not respond at all, or promised to write a review but never did. In the example below, that happened with anonymous referee #2. By the time the editor gives up on ever receiving a promised review, months may have passed.

If you give people a week to respond to an invitation, then three weeks to write a review, it may take a long time if many people do not respond to invitations quickly. Suppose 20% accepts a referee request and you want 2 reviews, then it makes sense to ask 10 people immediately, just to speed things up.

Answer 4

I've never seen this before so it doesn't seem common but an editor can solicit and pass forward as many reviews as he or she feels is necessary... Definitely not unethical.

Answer 5

Here is a more modern answer. The modern peer review system at Springer/Nature and Wilely allows the editor to specify the maximum number of reviews they'd like to receive (I usually choose 3 in case someone who agrees does not send one in).

In this system, the editor can then send an invite out to many people e.g. 5-10 at a time, even. Once 3 spots are filled, all other potential reviewers who have yet to accept the assignment are automatically notified that they are no longer needed. This prevents wasting 5 people's time but allows for faster acquisition of reviewers.