What are the ethics of reviewing a paper, spotting issues, not raising them in the review, and then trying to publish a paper on the issues?

Question

I had a paper that went through a long peer-review process (~1 year). One of the reviews seemed to obviously be from a specific individual who strongly disagreed with the paper because it critiques their work, although they didn't sign the review. Less than 24hrs after the paper was published, that specific individual posted a pre-print criticising the paper, which brings up some issues that were addressed during the review, but also several issues not mentioned during the review (e.g., a typo in one of the results tables). Speculatively, it seems like they were using our data/code and withholding issues they spotted to prepare their own pre-print while reviewing my paper. The pre-print even includes a link that only appears in the draft that was used in the review process, not in the published paper.

What are the ethics of this type of behaviour from a reviewer? It seems sketchy, but I'm not aware of official guidelines around this.

Edit: I did not post a pre-print of my own. So the first time my data, code, and results were made public was at the time of publication.

Answer 1

This seems very unethical as you describe it. I would write immediately to the editor, informing them of what has happened. The editor knows who the reviewer was, of course.

It is possible that you have the wrong individual in mind and the critique was generated outside the review process, especially if you have released a preprint.

But if your guesses are correct, it is pretty clearly unethical.

Answer 2

There is not necessarily anything particularly unethical going on here. Sometimes as a reviewer you will disagree with the arguments/methods/conclusions of a paper you are reviewing without there being anything objectively wrong about the paper that should stop the publication of this paper. Peer review ultimately is not the best place for an extended scientific debate.

In such a situation it is often a better idea for the reviewer to let the paper be published and write their own response to the paper. This seems to be what has happened here. Of course, technically what they should have done is wait for the final published version to be released and base their response on that. But, they can hardily be blamed for starting this after submitting their final review (presumably some time has past between their final review and publication). In any case, the preprint appeared after your paper was published. It is technically possible they simple stayed up all night to write the response in 24 hours. (Incidentally, avoiding these types of ethical gray areas is one of the reason that I only agree to review papers that are already available as preprints.)

You infer the reviewer has purposefully withheld comments from the review to base their own paper on. This it not necessarily the case. As an example you mention them remarking about a typo in your results, but it is at least plausible that they spotted the typo, only after submitting their final report. (Personally, I would consider pointing at typos in other people's work in a preprint the epitome of pettiness, but that is a separate issue.) They may also have concluded that some of their comments were irrelevant to whether or not your paper should be accepted for publication (in which case they would have no place in the referee report).

Answer 3

There are three separate issues here that I can see, and two of them suggest a problem with peer review in your case.

First, most (all?) reputable journals and conferences have policies requiring confidential peer review. That is- reviewers are not allowed to share the work or discuss the work with until it is published. This gives the author the opportunity to present their best possible work to the community, and prevents low-quality work from entering the academic discourse without the validation of peer review. If your work was shared outside of the peer review system, it indicates a major breach of protocol.

Second, most (all?) reputable journals also have policies about conflict of interest in peer review. At a minimum, all such conflicts need to be declared to the editors involved, and ideally anyone with a conflict should not be a reviewer. This is not always possible, such as when the topic concerns a very narrow specialty, but at a minimum it is a sensitive situation that the editors should have known about.

In order for this person to have gotten your work, it would seem that one of these two best practices was not adhered to. If your field is actually incredibly narrow in scope then perhaps the review was allowed to be conducted by the person you were critical of. It is certainly worth a letter to the editor and they certainly should not be surprised by it.

The third issue is the timing. This is not a problem, except as it implies a breach of protocol in points one or two above.

Once the work enters the public discourse then it is free game for comment. The person you are critical of did the right thing by delaying their publication until your work was published, and that is all you can really ask of anyone. If we are to have confidential peer review then there must be a point at which the restriction of confidentiality drops (in part because the reviewers are often the best qualified people to participate in the public discussion that follows the private review). You can see this especially with high-profile papers when they are published- all of the people who you think would normally be commenting are silent, then suddenly the paper is published and they all give a big sigh of relief and say, "OK, now I can talk about this."

As an author you might understandably want your work to have a "grace period" where it can be discussed freely without rebuttal, but as far as I know that's never been a thing. If you have to pick a point in time to drop confidentiality, then the public publication seems to be the best.

Answer 4

I am not sure the reviewer's behavior is unethical. Technically, they did not violate confidentiality as their preprint was published after publication of your paper.

it seems like they were using our data/code

But again, they did not publish anything before your publication. After that, a preprint using your data/code looks OK (if your work product is properly cited). They probably were preparing their preprint before your paper was published, but I am not sure this is unethical, as the preprint was published after your paper was published.

it seems like they were ... withholding issues they spotted to prepare their own pre-print while reviewing my paper.

I assume the reviewer recommended rejection of your article. As others wrote, the reviewer could just make arguments that they believed were sufficient to warrant rejection.

The pre-print even includes a link that only appears in the draft that was used in the review process, not in the published paper.

If, as I assume, the link is accessible to general public, providing it does not violate confidentiality either.

In general, you criticized the reviewer's work, they published an objection as soon as they could. I would say, this is understandable.

Answer 5

This is an interesting question and I think it depends on which ethical framework you are operating under.

If you subscribe to a consequentialist ethics framework, then you would judge the ethics of this reviewer's actions by what the consequences are likely to be. If the reviewer did notice an error in your paper, but chose to withhold that information from you, then the foreseeable consequences include a paper possibly being published which contains one more error than it would otherwise. That's a bad outcome for the wider pursuit of knowledge (and for you as an individual), and the outcome is a foreseeable consequence of the reviewer's actions, so the reviewer has an ethical duty to inform you of the error via the review process.

On the other hand, if you subscribe to a deontological ethics framework, then you would judge the ethics of this reviewer's actions by what rules the reviewer was supposed to be following, or what the reviewer's responsibilities were. The purpose of a review is to make a recommendation to the editor regarding whether the paper is suitable for publication in that journal. If the reviewer honestly believes that the paper is unsalvageable, that fixing every error would not result in a paper suitable for publication, then their ethical obligation to make a sound recommendation using their best judgement does not entail mentioning every error they noticed in the paper. Keep in mind that although the review is sent to the author and the author is expected to respond to it, the purpose of the review is to help the editor make an informed decision about whether to accept or reject the paper.

Note: I've answered by assuming the premise of the question, which is that the reviewer did notice these particular errors before submitting their review to the editor. It's quite possible that they noticed later, in which case the question is more hypothetical. There is still a related question of whether they have an obligation to inform the editor about minor errors they discover after submitting their review; I'm not sure what the standard practices and norms are in such a situation, but even a consequentialist might say that if the reviewer sincerely believes that the editor will reject it based on their recommendation, then as far as the reviewer knows, informing them (and you) of additional minor errors would be inconsequential.