23

Some years ago I published an article in a Copernicus Open Access Journal. They have a peer review process called Interactive Public Peer Review, i.e. the pre-print and the reviews are open access. Additionally, everyone can submit a comment on the pre-print.

I liked this openness, although I didn't get any extra comments other than those of the reviewers.

Some days ago another scientist told me that several scientists have concerns about this openess. They don't like the idea that their work can be commented by everyone.

I would like to know why there are concerns and how many scientists have concerns about this sort of peer review process.

Are there any studies or surveys answering this question? Are there any reports about the effects of this sort of peer review process?

Improve this question
2
  • 4
    I suppose the main point of contention here is whether the commenters are really going to be "peers" or not. I'm imagining if some random subject gets politicized and "goes viral", for example.
    – Daniel R. Collins
    Jan 26, 2017 at 23:45
  • 3
    Here's a very simple blog post on a neat trick in security: schneier.com/blog/archives/2006/01/countering_trus.html Now read the comments, and you'll see that despite the fact that the trick is relatively simple to understand, many people post that "it can't work", proving that they don't understand the technique at all. Now Schneier is a professional blogger and doesn't mind - and doesn't treat at as his job to correct all these misguided folk. But you can see why some people would be hesitant to expose their work to criticism from random people on the Internet.
    – DepressedDaniel
    Jan 27, 2017 at 3:07

2 Answers 2

Reset to default
7

This answer discusses a survey that indicates blind reviews are much preferred to open ones. The main reason, from that answer, seems to be

  • Many people don't want to review non-anonymously. I know many people on both sides of the fence on this issue. Once concern is that authors are often sensitive to criticism and may harbor ill-feelings against a person who gives an unfavorable review (especially if the reviewers have not properly understood the paper). Consequently, open reviewers may be biased (more so than blind reviewers) to colleagues they respect or want a good relationship with.

Fomite mentions a couple of other issues, but I see this as the main one (and it seems the survey does too). Some other potential issues are

  • Certain "rivals" or "critics" can try to ruin your career by giving unfairly harsh reviews.

  • Some well-respect person could make a comment based on a cursory read of the paper that ends up heavily influencing the reviews, and maybe no one reads the paper carefully enough.

Improve this answer
2
  • Thanks for the mentioned survey(summary), @Kimball. I discovered this article about the ACP publication process.
    – FuzzyLeapfrog
    Jan 27, 2017 at 15:17
  • Scholarly communication and peer review, March 2015. Commissioned by Wellcome, this report by the Research Information Network examines the current status and possible future direction for the peer review of research papers. On pages 22-23 they provide a good summary of the critical aspects of post-peer-review and comments.
    – FuzzyLeapfrog
    Jan 28, 2017 at 8:47
4

I don't know that there are any surveys looking at this, but some concerns I have, or have heard from peers, involve the following:

  • Open means open, which results in a way more expansive definition of "peer" than in other journals. This can range from the merely problematic (people outside your field ignoring field-specific convention) to crippling (mass commenting on controversial topic by trolls).
  • It's the antithesis of double blind peer review, which is also something that's being pushed fairly heavily. If you favor that approach, post-publication peer review is something of a problem.
  • Post-publication peer review is not fairly distributed. J. Random Student isn't necessarily going to get the same kind of comments as Professor Big Deal. It's possible they'll be different in volume, hostility, etc. Standard peer review, for better or worse, does ensure a set number of reviewers.
Improve this answer
5
  • The Coperinucus peer review process ensures a set number (i.e. two) of reviewers picked as in any other peer review process. Comments from additional (random) people are only on top. Additionally, the commenting process is not as easy as commenting on any random blog, so mass commenting and trolls won't happen that easily.
    – FuzzyLeapfrog
    Jan 27, 2017 at 10:28
  • @FuzzyLeapfrog Those are traits of a system - there are other open peer review systems that don't have the same properties.
    – Fomite
    Jan 27, 2017 at 10:29
  • Could you please privide some examples for (non-predatory) journals/publisher having a different (less quality controled) open peer review process.
    – FuzzyLeapfrog
    Jan 27, 2017 at 10:35
  • @FuzzyLeapfrog PLOS Current Outbreaks, where commenting is literally as easy as commenting on any random blog, what with using WordPress as their publication engine.
    – Fomite
    Jan 27, 2017 at 10:46
  • Thanks, @Fomite! Would be great to know how much "trolling" PLoS observes. Sounds like an interesting project.
    – FuzzyLeapfrog
    Jan 27, 2017 at 10:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .