How often should an editor ask for reviews from the same referee?

Question

There is a journal editor who keeps sending me review requests. He sends a new one shortly (a couple of weeks) after I return the previous one. I tend to accept because they align quite well with my expertise and I want to stay in good terms with this editor, who is a prominent figure in the field. I am a slow reviewer, but tend to give quite detailed and comprehensive reviews (compared with what I usually get for my own papers), and I guess the editor is happy with the quality of my reports, otherwise he wouldn't keep asking. However, I wouldn't mind if he asked less often. After all, this is only one journal and I also get review requests from other journals.

I have no experience as editor, and I am wondering if there are any best practices from that side. Is it common for an editor to keep "milking" that reliable reviewer? For those of you who have editorial duties, do you have any limit on how often to ask? Once a month, once every two months?

My intuition tells me there's a risk the reviewer gets tired of getting so many review requests from the same editor and the editor may end up losing that valuable resource.

Answer 1

From my experience as an editor, it is not unusual for referees to ask for "time off" from being assigned reviews and give bounds for the frequency of requests. As Buffy suggested, some editorial systems allow referees to enter this information and the editor sees it before sending out review invitations. I was glad to honor such requests from reviewers---I wanted them to feel satisfied & respected so that they would write more helpful reports in the future.

Similar to Fred Douglis, I tried not to ask anyone (other than editorial board members!) to review more than twice a year. However, we aren't mind readers. If your pattern is to submit a helpful report and then say yes to another request soon afterwards, then maybe you have the time, really enjoy reviewing, etc. What editor wouldn't follow up on such enthusiasm? My advice is that it's up to you to speak up about what you have time for, how often you're comfortable reviewing, etc. I think there's very little risk of getting on the editor's bad side by setting up those kind of parameters.

Answer 2

I'm not an editor, so I don't have the first-hand perspective you're asking for. Anyway, according to Publon's Global State of Peer Review 2018 report, ~10% of reviewers are responsible for ~50% of peer reviews (Nature News summary, full report), so some reviewers probably do get "milked". The report also discusses if there's a rising "reviewer fatigue", where review invitation acceptance rates are going down. That might make an editor more likely to repeatedly invite their more reliable referees.

Since your profile mentions you have a physics background, I'll mention something I've heard from APS editors. They try (across the journal family) to limit review requests a bit. First, they try to avoid asking a reviewer review multiple manuscripts at the same time, but do make exceptions for e.g. resubmissions. Second, they have a stated aim to give reviewers a two-week break between submission of the last report and the next invitation to review.

Answer 3

Even for reviewers I know well (and probably have worked with), I will try not to ask the same person to review journal submissions more often than every 4-6 months. I certainly wouldn't say "good job on that last one, here's the next one for you!"

But it all comes down to how people respond. If someone declines, I'm more likely to wait a long while before asking again, but if they agree whenever I ask, I can imagine that window narrowing. Especially when it often takes asking 8-10 people just to get 3 reviewers.

Answer 4

An editor should and normally does keep a database of reviewers. It contains more than just their email addresses. There is no particular reason why it can't contain preferences for frequency as well as the other information. So, an editor can and should ask, and try (hard) to honor those requests. Then the frequency is determined individually and need not be the same for all.

It would be fairly easy to write an application that will move potential reviewers forward or backward in a queue so that the process is automated, more or less.

Another system is to provide a list of papers to a set of reviewers and let them bid on them. This is harder to automate since editor judgement is probably needed to match papers with reviewing skills, history, and preferences. But a fairly large subset of the needed reviews can be handled purely via reviewer requests. Some CS conferences use this method. Then, fewer specific requests need to be made.

You are getting a lot of requests because (a) you accept them and (b) you do a good job apparently. You will keep getting them unless you ask for a pause or a slower or faster request rate. Accepting one two weeks after completing the previous one feels like service above and beyond the call of duty to me, actually. If it is burdensome in any way, or interferes with your own work, speak up.

Answer 5

The last time I handled a journal which had a standard review deadline of ~28 days, I had a personal limit of once every two months. It's not a strict rule, and if e.g. a paper is revised within two months I will still invite the same reviewer. This limit is also not based on any statistics or data - it's simply something I intuited. It was also rather rare for me to actually run into the limit since we don't usually get that many papers in only that single reviewer's expertise.

If you are getting too many reviewer invitations, absolutely feel free to decline. Editors see so many declined invitations that they are probably not going to bat their eye at yet another declined invitation. Just say you don't have time (which is true as well). There's a good chance you can also indicate in the journal's editorial management system periods for which you aren't available.

You don't have to feel bad about declining - there are thousands upon thousands of researchers in the world, and there is always someone else who can review the paper. If you can suggest someone else, it's not even that inconvenient for the journal to invite them.