15

This and similar questions ask how much time a researcher should devote to reviewing a paper, or to reviewing papers in general. But how much time do scientists spend refereeing papers as a matter of fact?

I am specifically looking for answers based on empirical statistics with a large-enough n, be it within one or across several disciplines. I'm not so interested in anecdotes or educated guesses.

  • 4
    At those who close-voted this: I completely fail to see how this is off-topic. – Wrzlprmft Dec 20 '18 at 16:28
  • 2
    @Buffy, I did (and shared) my research, as you can see below. I also want to tap your knowledge -- a reference request (see the tag). That's what a Q&A is about. As to this being off-topic, I can hardly imagine anything more specific to academia. Even research isn't more specific to academia than peer review, since lawyers, journalists, engineers do research, but don't write peer reviews. – henning Dec 20 '18 at 16:34
  • 4
    @Buffy I'm asking for an answer that's based on valid sources (as opposed to a long list of personal observations). – henning Dec 20 '18 at 16:39
  • 7
    @Buffy As far as I can tell, this question is perfectly in line with how the reference-request tag is supposed to be used. Perhaps all such questions could be strictly interpreted as "shopping for individual papers to read", but they don't tend to be. – Anyon Dec 20 '18 at 16:52
  • 2
    @Buffy: As for your first argument, I opened a respective Meta discussion, if I understand it correctly. – Wrzlprmft Dec 20 '18 at 17:02
13

Short answer: One day in a month.

In a piece on the "challenges and opportunities" of peer-review, Ware (2011, 25) estimates the median time spent per article to be 5 hours, and the average to be 8.5 hours. He acknowledges that these figures "vary by discipline and also by experience" (ibid.), but doesn't go into more detail. "Active reviewers" report an average of 14 reviews per year (ibid., 28). There is no explanation in the paper of what counts as "active" or how many papers the average academic reviews in a year.

Average time spent reviewing. Source: Waide 2016. CC-BY-NC-ND

However, in a follow-up publication, the same author defined active reviewers as those who review more than one paper per month (Ware 2016, 3). The same publication reports a follow-up survey with a slightly different methodology. Although the findings are therefore not directly comparable, their thrust is similar, with the average (mode) number of papers reviewed per month between one and two (ibid., 33), the median time spent on the last review at five, and the mean at 8.4 hours in 2017 (ibid., 34). The survey included more than 2000 respondents from different fields and demographics, with the arts and humanities somewhat underrepresented (ibid., 45).

Average number of articles reviewed. Source: Waide 2016, . CC-BY-NC-ND

Figures: Ware 2016. CC-BY-NC-ND.


References

Ware, M. (2011). Peer Review: Recent Experience and Future Directions. New Review of Information Networking, 16(1), 23–53. https://doi.org/10.1080/13614576.2011.566812

Ware, M. (2016). Publishing Research Consortium Peer Review Survey 2015. Mark Ware Consulting Ltd., Bristol, UK. http://publishingresearchconsortium.com/index.php/prc-documents/prc-research-projects/57-prc-peer-review-survey-2015/file

  • 4
    In a follow-up survey in 2015, Ware apparently found the same median time. There's a report here. There active reviewer is defined as those reviewing one or more papers a month. There's a lot of detail there about variation with field, with region, and with age (the mean time to review falls off with experience). – Anyon Dec 20 '18 at 16:21
  • 1
    @Anyon, thanks! Feel free to edit. I'll do it when I get around to it. – henning Dec 20 '18 at 16:26
  • You're welcome! However, I've been told that I'm on vacation and shouldn't spend time on lengthy edits or answers, so I'll have to leave that for you or volunteers. – Anyon Dec 20 '18 at 16:53
  • 4
    @Anyon If active means reviewing at least 1 paper a month, I think most pure mathematicians would not count as active reviewers, which may skew the stats a lot for math. – Kimball Dec 21 '18 at 16:20
  • 2
    @Kimball I agree - seems like a high bar for physics too. I wouldn't count, for example. More importantly, since time is finite it probably skews the stats towards those who spend less time on a single review, regardless of field. – Anyon Dec 25 '18 at 9:24

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.