Ethics of publishing received peer reviews

Question

What are the ethics of publishing (on the web, etc.) peer reviews received for a paper? Does your answer change if the paper was accepted or rejected?

I am specifically interested in the "one-shot" case typical of computing science conferences, without rebuttals, where there is no or limited dialogue between an author and the reviewers.

This is quite distinct from Can I publish the reviews I write? as here I am talking about reviews I have received, not those I have written - the reviews' authors retain their anonymity, and presumably the paper would be included alongside those reviews.

Vijay's response below includes a summary of much of the other responses and my comments on those.

Answer 1

Let me add some data points to this discussion. Some items below are about discussing reviews and editors in a public forum and not about publishing reviews.

• Jens Teubner makes available the reviews to his papers online. I do not know him but I have heard from someone who does that he said he has not received negative feedback about doing this. Maybe you can ask him for more information.
• Doron Zeilberger has published reviews in Opinion 87 and for his paper Automatic CounTiling.
• Doron Zeilberger's Opinion 61 is about rejections and accountability (it has a response from Luca Trevisan) and his Opinion 81 is about rejection and snobbery.
• Peter Clark's mail to Doron Zeilberger about some material Zeilberger published online. This is only tangential to your question, but I think it's good to keep in mind that when you go down the road of open publication, you should be ready for others publishing material about you or your reviews.
• The Writings of Leslie Lamport chronicles in very direct terms the stories behind his papers including some notes about editors and reviewers (for example papers 62, 122,129, ).
• In The Writings of Leslie Lamport Paper 132 he talks about having written an 'unkind review'.

I have been meaning to publish my reviews for multiple reasons and I am glad to see that other people have been thinking the same and some have done it. I do not think the result or answer should depend on whether the paper is accepted or rejected. If reviews are published, I believe that one should also publish the version of the article that was used to make the reviews. Otherwise, the reviews are like quotations taken out of context. If there are coauthors, one should obtain their permission first, or at least include a disclaimer that you are publishing your reviews on your behalf only. As an example of a disclaimer, see Lamport's page:

Some of the stories read like complaints of unfair treatment by editors or referees. Such cases are bound to arise in any activity based on human judgment. On the whole, I have had little trouble getting my papers published. In fact, I have profited from the natural tendency of editors and referees to be less critical of the work of established scientists. But I think it's worth mentioning the cases where the system didn't work as it should.

The notification of acceptance is usually signed by the editor of the venue, even in the case of anonymous review, so one should ideally obtain their permission if you will reveal who the editor is. The same applies for reviewer permission in signed reviews. I cannot tell whether people in the links I give above obtained editor permission first so there may be precedent for not doing so. Publishing reviews intended for private circulation still takes them out of context. I would add a disclaimer that the reviews and notification letter were written as private communication and if published without permission of named entities, I would note that too. The latter is in case your intention is to protest the status quo by subverting standard conventions.

I haven't answered your question because I don't have a clear answer. Publishing reviews is not conventional academic behaviour. Doing so can be construed as unprofessional depending on how you publish them and what additional commentary you add. Calling it "unethical" seems a rather heavy handed judgement to make. There are things an anonymous reviewer can do that are clearly unethical (steal research, suppress publication, circulate the manuscript) or questionable depending on context (force citations, comparisons, reject without reading, write ad hominem reviews etc.). There is very little an author can do to wrong a reviewer that is remotely comparable and publishing a review does not seem remotely on that scale to me.

There are multiple reasons to publish reviews including accountability for all parties involved (including authors) and as a form of protest. I believe this intent is important to consider because forms of protest do subvert what may be considered acceptable behaviour. A useful thought exercise might be to put yourself in the reviewer seat and ask if you would be fine with similar treatment. I would not have an issue if the reviews I have written were made public (even if I knew that stylometric techniques could be used to identify me). I do not think all reviewers feel that way.

Finally, let me point again to Jens Teubner's page, which comes across to me as a model of how to publish reviews while retaining professionalism and dignity.

Answer 2

In the journals where I have been invloved as editor, author or reviewer (except in ones where the review process is open on the web), the implicit understanding has been that the communication is closed between the author-editor-reviewer. I have not seen any explicit rules stated by these journals to control against any such public dissemination but I have not looked for it either. I think that, particularly in cases where it is not explicitly stated that reviews are public, posting reviews without consent from the reviewer would be similar to publisihing someone private letters wihout consent. It is not illegal but ethically very distasteful. I do not think it matters whether or not the review was anonymous. Whether the paper was accepted or not would not matter either. I would consider posting without consent just as bad under all circumstances.

Answer 3

As argued in the answer by Peter Jansson, it is ethically not acceptable to publish peer reviews that you receive for your paper, and I agree with that.

But there is also a legal aspect to it. The review is an intellectual work and as such will typically be subject to copyright. And that holds even if you don't know the author. So publishing a review will be a copyright violation, unless you get permission from the person who wrote the review.

If you want to highlight bad review practices, instead of just publishing the review consider citing from it. The actual difference may be marginal, but from both a legal and an ethical perspective, you should fare much better with this approach.

And my answer doesn't change depending on whether the paper is accepted or not.

Answer 4

If the question is "Should the author publish the reviews he got for his paper(s)?", then I'd say no (maybe with some exceptions, but this would have require a per-case discussion). I believe the other answers have more or less covered that.

But, if the question is "Should the journals publish the reviews?", then I believe it would be both ethical and useful to publish the final positive reviews, and I have several reasons for this.

First, giving a positive review is like giving a positive grade on an exam. The person doing so should stay behind his "verdict" with his professional reputation. The review is one of the results of the work that the researchers do, so publishing positive reviews doesn't seem to me much different from publishing the results of the research in papers.

Second, and quite related to the first, I've read some really crappy papers, with nonsense, obvious errors, misquotes, etc. Reviewer cannot "catch" everything, but some of the papers get bad enough that it is obvious that the reviewer didn't do his job. If the reviewers knew their names would forever be publicly associated with such paper, I believe some of these might actually try to do their job.

As for the negative reviews, I see no point in "shaming" the author if his paper was too bad (in whatever way) to be published. If this was not the case, but the reviewer is to be "blamed" (i.e., for misunderstanding the paper), the issue can be resolved with the editor, or the paper can be submitted elsewhere, again giving no reason to make the negative review public.

One might argue that the negative review is also like publishing the results of the research in papers, but I see it more like a failed research, which is not something that one usually publishes.

Answer 5

The ethics are the same as publishing any other communication that is assumed to be in confidence: if the content contains critical information about an illegal activity, you may (depending on the nature/severity) be morally obligated to turn it over as evidence to an appropriate party; if it is otherwise urgent or critical for others to be aware, you may be morally encouraged to find a venue to publicize it (as "whistleblower"), and in pretty much all other cases, you should keep in private.

You can complain to editors and others responsible for the conference or journal; you can commiserate with colleagues and try to find out whether this is systematic or not. But trying to shame reviewers in public is very unlikely to accomplish any valuable goal. (It may provide a satisfying revenge at the cost of other of your goals; I encourage you to think very carefully before deciding that this is worth it.)

There are all sorts of reasons you might have a bad review (covered in other answers); assuming one that makes you the most angry and/or feel most self-justified is a very natural reaction, but probably isn't the best way to a productive conclusion.

Instead, when you find things like this, you are probably much better off applying to other conferences, publishing in other journals, and maybe quoting (anonymously) the worst parts of the review in a blog or somesuch explaining why you're no longer going to whichever conference or considering publishing in whatever journal. There are options in some fields for publishing with open reviews (the Frontiers life science journals, for instance).

Publishing the full review and naming names is a good way to make people angry, make other reviewers not want to review your work, and to not get anything to change. Adding your voice (and papers!) to existing movements that seek to improve the peer-review process is much more productive, even if it's not as viscerally satisfying.

To be completely clear about the ethical implications: you are breaching the trust of the editor(s)/organizer(s)/reviewer(s) that reviews are confidential information. Breaching trust makes people less likely to trust you in the future, and is (in most ethical frameworks, including intuitive ones) unethical unless perhaps there is a very compelling case why this must be done. In your situation you have not articulated such a compelling case, especially given that are a variety of other avenues to take if you're actually concerned with the quality of the reviews. If you are working in academia, presumably you know how to quote tiny sections of a full work to make your point--that's much more acceptable.

Answer 6

As said before, publishing reviews breaks convention but this, in itself, does not make it unethical. Publishing anonymous reviews does not really affect reviewers—reviewers could do much worse to authors than what authors could do to reviewers. (See Vijay's answer for examples.)

In my opinion, the confidentiality of reviews is to protect the author: the fact that a paper got submitted remains confidential prior to publication. Thus, authors may considerably rework their paper after receiving peer reviews, resubmit to another conference or even completely abandon it.

Reviewers are protected by their anonymity -- such that a young researcher can openly state his opinion about the paper of a renowned academic without fearing bad consequences. I'd assume even stylometric analysis cannot reveal a reviewer's identity without doubt.

Despite all that, publishing reviews might still be regarded as unprofessional, depending on circumstances and context.

Answer 7

This might be related to your question: http://www.peerageofscience.org/

The concept is that 1) you peer-review your paper before submitting it to a journal, 2) peer-reviews are peer-reviewed, 3) participating journals send you publication offers if they like your paper, 4) you may submit your peer-reviewed paper to any journal you like

I'm not sure are the reviews public themselves, I did not use the site yet, but at least there are peer-review scores appearing on the main page