43

This is a follow-up question to Open versus Blind reviewing process, and is somehow related to What happens to the reviews that people write for journal articles after they're sent back to the author?

However, my question does not concern the reviews I receive for the papers I submit, but concerns the reviews I write for papers I have been assigned. Since the whole process is done under confidentiality, it is not clear who owns the copyright on a review I wrote, and what does the review include.

For instance, let us assume that X is member of a PC of a conference Y, and assigns to me a review of a paper Z, written by A and B (assuming it's not a double-blind). Can I publish on my blog: "Here is my review for Y, asked by X, on the paper Z, written by A and B"?

I think that there are two points here:

  1. Is it legal? (for instance, publishing the camera-ready version of a paper might be illegal due to the copyright transfer, would it be also the care here?).
  2. Is it ethical? (who should I ask in order to do so? X? Y? A and B? everybody?).

EDIT: There is been several comments/answer wondering why I would like to publish a review I wrote. To give a bit of background of this question, I believe that the current reviewing system, created when the academic community was small and there was no Internet (i.e. no easy access to information), might not be the best, although clearly working. This is for me a very interesting debate, but somehow out of the scope the Q&A format of Academia SE, which is why I tried to focus on my question on whether it was possible to do so, not if it was the best thing to do in the current system (and just to be clear, I don't plan to do it, but I just like to know what are my options). Anyway, thanks for the answers bringing a different light on this debate.

EDIT 2: After seeing the update in Jeff's answer, I just realised that I didn't make it explicit that I was talking about reviews after the reviewing process. Jeff says that it's ok if the paper is accepted, and although I clearly understand the argument of why I shouldn't publish a review of a rejected paper, the question still holds: by publishing a review of a rejected paper, I publish the information that these authors submitted this paper to this conf/journal, which is supposed to be confidential between the authors and the editors. Would I break any rule by doing so?

  • 14
    Yes. Publishing confidential information breaks Wheaton's Rule. – JeffE Mar 9 '12 at 8:46
  • 7
    Another thought: if you want more credit for public reviews, why not review books? William Gasarch, for instance, maintains a list of books he needs reviewed for SIGACT News -- a great way to get free books, by the way ;-) A similar option surely exists for other (sub)fields. – Anthony Labarre Mar 9 '12 at 9:45
  • @JeffE Well, I wouldn't want to break Wheaton's rule, I guess I would be banned from Reddit then! :) – user102 Mar 9 '12 at 10:03
  • 16
    my counter-question would be: Can I publish the sloppy reviews that I receive? That would attract a large audience of frustrated academics :) – ElCid Aug 28 '12 at 11:40
  • 3
    @ElCid You can make papers out of them: Friston (2012) Ten ironic rules for non-statistical reviewers. Neuroimage, 61, 1300-10. – Mark Jan 23 '18 at 17:50
53

The standard rule in my community is that once I finish reviewing a paper, I'm supposed to pretend that I don't know the paper exists. In particular, I am not supposed to use any insights I gained by reading that paper in my own research. I am not supposed to reveal the results to my colleagues. Some venues ask that I destroy any copies of the paper I'm reviewing, along with any programs or data I used to verify the paper's results. This embargo lifts only when the paper is finally published, but I am never supposed to reveal my identity to the authors, even indirectly.

Under those rules, publishing reviews is completely unethical. Maybe it would be okay if I had the explicit permission of both the authors and the editor, but I would expect most authors and reviewers to vehemently object. I would feel weird even asking.

But even under less stringent reviewing rules, I think posting reviews is a very bad idea. Criticism is best given privately. One of the purposes of anonymous reviewing is to give authors brutally honest feedback on their work. Referees can offer direct criticism without worrying that it will harm the authors, and authors are more willing to hear that criticism because they know it will never be public.

Yes, that means authors sometimes get credit for ideas that I suggest in referee reports. (Most authors are nice and thank the anonymous referee.) On the other hand, several referees have offered suggestions that have significantly improved my papers, so it all comes out in the wash.

Update: I should add that these ethical constraints attach only to reviews of unpublished papers being considered for publication. Once a paper is actually published, everyone is free, if not encouraged, to publish their own reviews of the published version.

  • 4
    Thanks for your answer, and to be honest, I wouldn't think about publishing very negative reviews, and ideally, I'd rather like the authors to publish my review with their papers if they think it can bring something. For instance, a reviewer once found a very tricky bug in a complex program I attached to a paper. Founding this bug probably took him a lot of time, and it improved a lot my program. The only credit I can give him, is to thank the "anonymous referee". I personally don't think it's fair, but I don't want to impose my view over everybody :) – user102 Mar 8 '12 at 18:17
  • 1
    You can't reveal you identity even after the paper was published? – Artem Kaznatcheev Mar 9 '12 at 3:53
  • 2
    @CharlesMorisset Why isn't it "fair"? Just pay it forward. – JeffE Mar 9 '12 at 8:10
  • 2
    @JeffE I perfectly understand the idea that it works globally because of "today you, tomorrow me", but ideologically speaking, I disagree with it, I don't think anonymity improves the system, on the contrary. And I'm not saying it's not fair, just that I don't think it's fair, and if I could, I would like to incude the name of this reviewer in the paper. – user102 Mar 9 '12 at 8:32
  • 21
    No, you misunderstand me. Once the paper is published, you are free to write a fresh review of the published version, just like anyone else seeing the paper for the first time. You are still not free to yell out "I was the anonymous referee that they thanked!" – JeffE Mar 9 '12 at 8:50
7

To the best of my knowledge, there are no copyright issues related to reviews you write, and I don't see who would own rights on them save you should those be applicable. To be fair though, I would ask at least X and Y whether they have a problem with you making the review public.

That said, I fail to see the point of making them public. Those reviews are of extremely restricted scope; if the paper was rejected, no need to rub it in by letting the rest of the world know how bad it was. Otherwise, as pointed out by Lars, your comments may not be valid anymore if they were addressed by the authors, but even if they weren't, I don't see the point in publishing the review.

... unless you want as many people as possible to know how thorough a reviewer you are so that they can send you more papers to referee?

  • 11
    Add a line to your CV saying "Reviewed for [journal/conference]". Under most circumstances, that's all the public credit you get. Don't worry, the editor will really appreciate your efforts, and may even be willing to say so in a recommendation letter. – JeffE Mar 8 '12 at 13:10
  • 5
    @CharlesMorisset "take the full credit for some work I do." okay, but who cares? I'm not sure who are you trying to impress. Editors? Other scientists? You'll impress them infinitely more by writing good articles of your own. And isn't reviewing journal articles part of the job of an academic? It's assumed that you're putting good work into it. This kind of strikes me as wanting special credit for showing up to work on time every day even though everyone else is too. – Amy Mar 8 '12 at 17:53
  • 1
    I was trying to ask who your target audience would be by publishing your reviews. (and in practice, is there a difference between "taking full credit for work" and "impressing someone"?) – Amy Mar 8 '12 at 19:07
  • 5
    "why can't I have this information for reading a paper, especially when I know that this information exists?" — But it doesn't exist. Your review refers to a version of the paper that is not published, which by definition means the authors have not made it public. By publishing your review, you are releasing information about the author's previous version that they may prefer to keep quiet. ("for potential recruiters, as another way to evaluate a candidate") Speaking as someone who has served many years on faculty recruiting committees, I would view this VERY negatively. – JeffE Mar 9 '12 at 8:16
  • 3
    Also, I don't think @Amy needs to apologize for her tone. – JeffE Mar 9 '12 at 8:18
5

Some conferences/journals say on their website that reviews should be treated confidentially, although I'm not sure if you're legally obliged to follow this. I would certainly ask everybody involved (i.e. PC and the authors of the paper) before publishing the review.

It's also possible that (part of) the review isn't valid anymore if you're raised concerns that the authors have addressed in the final version.

  • But when they say that reviews should be treated confidentially, do they mean that authors should treat the reviews they received confidentially, or reviewers should? I know that some people publish the reviews they receive for papers as some kind of public token of the quality of the conference (I once found the website of a person explaining why he was doing it, but I can't remember it :(), so I can understand the warning for the authors, but does it hold for the reviewers too? Do you some examples of the conf/journals, so that we could check what they say exactly? – user102 Mar 8 '12 at 8:53
  • 8
    Legally obliged? Probably not. But if someone asks you to keep something confidential and you don't, you're a jerk. – JeffE Mar 8 '12 at 13:07
  • 2
    @CharlesMorisset, a quick search brought up Maney and PLoS as two publishers who say that. Note that neither refers to the reviews themselves explicitly, but I would certainly assume that they mean them as well. I would also assume that the confidentiality extends to everybody involved in the process. – Lars Kotthoff Mar 8 '12 at 18:05
5

I feel that everyone is entitled to their opinions and their right to share them, and I agree that openly discussing published work is good for the scientific community. But I would tread lightly when it comes to papers you have personally reviewed.

If you have a serious issue with a publication, or have important comments to make after publication, the venue to do that is in letters directly to the journal, which they may choose to publish.

If you're thinking about publishing comments on your blog, consider what the goal of your blog is and who your audience is. Are you trying to open up a dialog about the work? Keep a collection of everything you've done? Advertise your expertise as a reviewer to editors?

If the goal of your blog is discussion of current work in your field, consider writing about papers that you have no connection to (groups you don't collaborate with, papers you haven't reviewed). Then you'll avoid any sense of bias or potential liability/ethical issues.

If the goal is to keep a collection of your work, you can do so privately and share in instances when you are trying to demonstrate your critical thinking skills (editors, job applications, maybe elsewhere?).

I think that overall the review process exists mainly, if not solely, for the benefit of the authors - not for the benefit of the reviewer or the scientific community at large. In the end, everyone gets the benefit of (hopefully) a better paper.

5

I would never publish a review as a review. I agree with other the other comments that the ethical standard is that you are meant to flush your brain of any insight resulting from what you have reviewed until / if it is published. However, this is certainly not what happens in the real world.

  • I've certainly seen and heard about people who actually published on the topic of a paper they reviewed before that paper was accepted anywhere. Some people just cheat.
  • Brains are not something you can flush. In fact, your mind is an academic's primary asset.

Personally, I think that if you have learned something or realised something as the result of the reviewing process, you should probably put it in the bottom drawer for a little while to give the author a chance to get it out and for you to be able to cite them, but ultimately it's an academic duty for ideas to be shared, not lost.

There are actually some famous stories of established academics reaching out to bright newbies and helping them publish ideas after seeing a paper in review, e.g. John Maynard Smith and George Price. These are sometimes seen as scandals, but provided that the original paper wasn't rejected specifically for the purpose of bagging an authorship, I think that is actually a lot more ethical than going away and doing the work without the author who inspired you, which sadly I think is more common.

5

I know answering my own question is a bit weird, but I just came across the website http://www.reviews.com/, and since I wasn't aware such sites existed, I thought it would be good to mention it for future references.

I haven't gone through the entire website, but it seems that anybody can register and submit reviews to papers (including conference/journal papers, and not only books), and these reviews are then available. I've just noticed (it's probably not new though) that for instance the ACM Digital Library has a tab "reviews", which references the corresponding reviews.

So, if I try to sum up the other answers, technically, it seems to be legal to publish a review, unless the editor made clear that all reviews must be treated as confidential, but as long as the article is under submission, it's clearly non-ethical, and would be quite damageable. However, once the paper is published, it's perfectly possible to publish a non-anonymous review (for instance using the website I mention above, but I'm sure other solutions must exist), that can of course be based on the actual reviewing process, but it should not be stated that the author of the review was an anonymous reviewer. Note that it's possible to publish "negative" reviews (for instance: this one, which might be behind a paywall).

Thanks for to all the answers and comments!

  • 3
    I'd add that you shouldn't say anything about the pre-publication version you saw, only the published version. (The exception would be if there was evidence of fraud, and I'd recommend going to the editor first.) – Ellen Spertus Mar 11 '12 at 21:50
  • 1
    This is certainly reviews of published works, not reviews done as part of the evaluation for publication. – vonbrand Sep 14 '15 at 18:12
2

Given recent movements towards open peer review, and that I broadly disagree with previous answers, I thought it worth adding another answer to this question.

Firstly, regarding the legality, you will likely find it completely within your rights to share the review as you please. Most journals mention 'confidentiality' in some respect, but are usually very unclear about what this means. The question of who 'owns' the review is currently being discussed by COPE, but I would suggest that the answer is obviously the reviewer.

Secondly, regarding the ethics, I think you have a duty to share your reviews wherever possible. There is a trade-off between Wheaton's Rule and your duty to the scientific process (particularly if you are a publicly-funded researcher), and this hasn't previously been raised in this discussion. Where you find the balance will for now be subjective, as no clear consensus has developed. Personally, I believe:

  1. Researchers have a duty to expedite the scientific process.
  2. A publicly-funded scientific process should be transparent.
  3. Authors, reviewers and editors should all be accountable for their actions.
  4. Reviewers deserve recognition for their contributions.

These points are particularly relevant if a reviewer makes suggestions that are not heeded by the author, in which case important insights might be buried. The COPE guidelines for peer reviewers suggest the maintenance of confidentiality. However, my interpretation is that this should only apply until the pre-publication peer review process is complete (i.e. until publication).

There are now more formal options for sharing your reviews; for example via Publons. This is my preferred option for sharing because reviews are not visible until the article is published (which I think ensures fairness to authors) and publishers have the option to hide the content of your review (which I think ensures fairness to publishers/editors).

0

Like elaborated by @ChrisSampson87 in another answer, I totally support the sharing of provided peer reviews after publication. I believe transparency can only add to speed and quality of scientific research, and most of the major flaws in peer review rely on the fact that it is kept secret behind curtains by most of the community. Likewise I am in favor of signing my reviews -- this is not always easy, and often leads to a backlash, which is halfway gone towards sharing it later.

When one signs a review and plans on sharing it later, immediately a greater sense of responsibility is created. There are websites allowing for this practice which implies that any member journals include editors who are OK with the practice, and authors submitting papers to those journals should be aware of the possibility of greater exposure after their publication.

Minor personal conflicts and egoistic feelings aside, I strongly believe bringing in more light to peer reviews, even of rejected papers, would do far greater good than damage to the working conditions of scientists everywhere.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy