56

I am an Assistant Professor of Chemistry. I try to be an active peer-reviewer but on average it takes a few hours to review a manuscript (reading, some searches in the literature, checking some relevant reference, and finally writing the report).

I review 2-3 papers per month and I need to spend some time outside my working hours. In various blogs, articles, websites like publons.com, I see it is not uncommon to review 10-20 papers per month.

Then, I came to the conclusion that I am doing something wrong. Because by my calculations, it is impossible to peer-review many papers along with various tasks of an academic.

11
  • 69
    Interesting. Based on the facts you are stating, I would say that someone reviewing 10-20 papers a month IS doing something wrong.
    – Emilie
    Commented Nov 21, 2017 at 15:28
  • 29
    Have you never received any of those reviews that clearly show the reviewer did not read the manuscript? And then been rejected based on them? :)
    – user541686
    Commented Nov 21, 2017 at 18:01
  • 14
    In my experience, the professor assigns the paper reviews to grad students.
    – jamesqf
    Commented Nov 21, 2017 at 18:23
  • 48
    it takes a few hours to review a manuscript — Wow. Chemistry papers are short!
    – JeffE
    Commented Nov 21, 2017 at 19:30
  • 18
    @JeffE: It's not the length of the papers; it's the nature of the review. Reproducibility is a lot harder in chemistry than in CS/math. You have to judge if the results are consistent with the claims of the "experimental" method, and the conclusions are consistent with the results.
    – aeismail
    Commented Nov 21, 2017 at 19:46

5 Answers 5

79

I don't think people who review 10–20 papers per month are likely to be doing a good job. They are probably just reading a paper once and writing a couple paragraphs (at most) based on superficial impressions. I've certainly gotten reports on my own papers that seemed to be produced that way; and they were both frustrating and not very useful.

I personally review about 3–6 papers each month, and that is by far the most of anyone in my department; I have also received outstanding referee rewards from multiple journals, mostly just recognizing my high levels of work. (This also represents a big part of the professional service work that is expected of me as a faculty member.) It would certainly be possible for me to referee five times as many papers (although I would have to contact a lot of editors to volunteer my services), but that would turn refereeing into a the largest component portion of my total workload. It would not be possible to do those reports properly and still be a productive researcher.

1
  • "writing a couple paragraphs" That's optimistic. Commented Jan 28, 2021 at 10:35
32

Not all reviews are created equal. A review that is based on a careful reading of the whole manuscript, and gives specific, actionable recommendations about how the paper should be changed, with clear justification, is far better than a review like a movie review that simply throws derogatory language at the paper (some highlights from reviews of my own papers: "shallow", "weak", "very, very bad") or rubber-stamps the paper despite serious problems. I think it's often worthwhile to spend time checking calculations and checking that cited papers really say what the authors claim they say, although such things are beyond the call of duty in most cases.

But, there's little incentive for writing good reviews or punishment for writing bad ones. All reviewers get out of reviewing is getting to list in their CV what journals they've reviewed for. So, inevitably, careerists write a lot of sloppy reviews. Also, as mentioned in a comment, unscrupulous professors may take the credit for a review written by a graduate student.

What I would recommend is, don't worry about how many reviews you're completing. Spend the time you think is necessary to write good reviews, and if that only gives you time to write a few, then good on you for putting in all the effort to help those papers achieve their potential.

For what it's worth, I'm a postdoc in public health and psychology and I review maybe 1 paper a month tops. Each takes me at least a day of dedicated work.

1
21

I know that it is common practise in some institutes that PhD students ghostwrite the reports. In this way it is easy to produce a large number of reports per month (in the same way that people produce a ridiculous amount of papers, grant proposals etc.)

1
  • 8
    Now it makes sense. That guy he mentions doing 20 reviews per month? Actually, perhaps it means that guy's graduate students and postdocs are doing 20 reviews per month...
    – GEdgar
    Commented Nov 22, 2017 at 12:15
4

Sometimes people who are part of the technical program committee of a conference are attached to several papers as "primary" reviewers. For example, in this conference, where I am in the TPC I am being asked to review 10+ papers due to an unprecedented number of submissions. The concession is that on some we will be allowed to just write a meta-review.

-1

In my opinion the peer review process is deeply broken. The quaility of the reviews I got so far is concerning. In most journals just one out of three was able to find a real flaw. Sometimes the other 2/3 criticise even positive aspects because they did not read the paper or lack complete understanding. Then you have to write a boring response letter back until the stuff gets submitted. In my opinion an exhausting process which often ends up with the fact that another group working on a similar process gets your data and submits urgently. I am pro non-anonymous reviewing. I guess the review quality would increase greatly and the reviewers would be also more polite sometimes.

I guess you have to choose a speed which fits to you. If you feel not confident to review 20 papers then do not do so.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .