42

I had a paper that went through a long peer-review process (~1 year). One of the reviews seemed to obviously be from a specific individual who strongly disagreed with the paper because it critiques their work, although they didn't sign the review. Less than 24hrs after the paper was published, that specific individual posted a pre-print criticising the paper, which brings up some issues that were addressed during the review, but also several issues not mentioned during the review (e.g., a typo in one of the results tables). Speculatively, it seems like they were using our data/code and withholding issues they spotted to prepare their own pre-print while reviewing my paper. The pre-print even includes a link that only appears in the draft that was used in the review process, not in the published paper.

What are the ethics of this type of behaviour from a reviewer? It seems sketchy, but I'm not aware of official guidelines around this.

Edit: I did not post a pre-print of my own. So the first time my data, code, and results were made public was at the time of publication.

8
  • 1
    Maybe it was a reviewer that submitted a review, then the editor rejected the review and went on to look for another reviewer? It would be worthwhile to contact the editor where you submitted the paper, by telephon (do not leave a paper trail for now), it seems the reviewer did not break copyright agreement&so, but its behavior is questionable. It may be that they hold a grudge, not with you but with the journal's editors...
    – EarlGrey
    Jul 1 at 12:34
  • 1
    @EarlGrey, why not a "paper trail"?
    – Buffy
    Jul 1 at 12:37
  • @Buffy you never know who is the editor friend of, maybe the reviewer in discussion here is one of the biggest contributor to the journal and a well-known expert (as well as a jerk)
    – EarlGrey
    Jul 1 at 12:40
  • 4
    @Buffy you don't know you are on the wrong end of a conflict, until you discover you are on the wrong end.
    – EarlGrey
    Jul 1 at 12:49
  • 5
    specific individual posted a pre-print criticising the paper, which brings up some issues that were addressed during the review - Are you saying some of the critique objectively does not apply to the published version of your paper?
    – Kimball
    Jul 4 at 2:39

5 Answers 5

46

This seems very unethical as you describe it. I would write immediately to the editor, informing them of what has happened. The editor knows who the reviewer was, of course.

It is possible that you have the wrong individual in mind and the critique was generated outside the review process, especially if you have released a preprint.

But if your guesses are correct, it is pretty clearly unethical.

7
  • Thanks for the response. Are you perhaps aware of any official "peer-review guidelines" that I could direct the editor to?
    – user72716
    Jul 1 at 12:43
  • 9
    The editor, if established, will be experienced in these things. They won't need "pointing".
    – Buffy
    Jul 1 at 12:44
  • 3
    you might find something in 'COPE Ethical Guidelines for Peer Reviewers' publicationethics.org/resources/guidelines/…
    – L_W
    Jul 1 at 21:26
  • 1
    Suppose the originator of the paper has a history of rejecting or ignoring reviews, or the reviewer has experienced their review of other papers being ignored or rejected. It would be logical to convert the review to a critique as the originator would then, in my view, be acting unethically in wasting reviewers' time.
    – Magoo
    Jul 2 at 13:09
  • I appreciate your knowledge. I agree with your conclusion but I am not sure exactly how did this conclusion is reached. What specific action was unethical and what specific code of conduct is violated? Like COI or confidentiality?
    – High GPA
    Jul 5 at 0:28
18

There is not necessarily anything particularly unethical going on here. Sometimes as a reviewer you will disagree with the arguments/methods/conclusions of a paper you are reviewing without there being anything objectively wrong about the paper that should stop the publication of this paper. Peer review ultimately is not the best place for an extended scientific debate.

In such a situation it is often a better idea for the reviewer to let the paper be published and write their own response to the paper. This seems to be what has happened here. Of course, technically what they should have done is wait for the final published version to be released and base their response on that. But, they can hardily be blamed for starting this after submitting their final review (presumably some time has past between their final review and publication). In any case, the preprint appeared after your paper was published. It is technically possible they simple stayed up all night to write the response in 24 hours. (Incidentally, avoiding these types of ethical gray areas is one of the reason that I only agree to review papers that are already available as preprints.)

You infer the reviewer has purposefully withheld comments from the review to base their own paper on. This it not necessarily the case. As an example you mention them remarking about a typo in your results, but it is at least plausible that they spotted the typo, only after submitting their final report. (Personally, I would consider pointing at typos in other people's work in a preprint the epitome of pettiness, but that is a separate issue.) They may also have concluded that some of their comments were irrelevant to whether or not your paper should be accepted for publication (in which case they would have no place in the referee report).

7
  • 3
    As far as subjective/opinion-based criticism is concerned, this answer makes sense. But the situation in OP's case contains an example for an objective error (typo in a table). Jul 1 at 15:05
  • 7
    @lighthousekeeper: But I don't think we (or OP) can know whether the reviewer might have spottet this specific typo only much later after their review had already been submitted. Jul 1 at 18:10
  • 1
    The fact that the criticizing paper was submitted within hours of the publication of the original seems suspicious to the least, since there was no preprint submitted. It is very implausible for someone to read, analyze, review in-depth an article, and then write a well-sourced, in-depth critique about an article within 24 hours of it's first public appearance ever.
    – Neinstein
    Jul 4 at 11:05
  • 3
    Also, even if the reviewer hasn't purposefully withheld issues, there's a huge conflict of interest in using the content of an unreleased article he is reviewing as the base for his new publication. I don't see room for any ambiguity in that there's a big ethical issue here.
    – Neinstein
    Jul 4 at 11:07
  • 1
    @lighthousekeeper It could be the case that the reviewers spotted these typos/mistakes after submitting their reports -- I sometimes do this too.
    – High GPA
    Jul 5 at 0:30
4

There are three separate issues here that I can see, and two of them suggest a problem with peer review in your case.

First, most (all?) reputable journals and conferences have policies requiring confidential peer review. That is- reviewers are not allowed to share the work or discuss the work with until it is published. This gives the author the opportunity to present their best possible work to the community, and prevents low-quality work from entering the academic discourse without the validation of peer review. If your work was shared outside of the peer review system, it indicates a major breach of protocol.

Second, most (all?) reputable journals also have policies about conflict of interest in peer review. At a minimum, all such conflicts need to be declared to the editors involved, and ideally anyone with a conflict should not be a reviewer. This is not always possible, such as when the topic concerns a very narrow specialty, but at a minimum it is a sensitive situation that the editors should have known about.

In order for this person to have gotten your work, it would seem that one of these two best practices was not adhered to. If your field is actually incredibly narrow in scope then perhaps the review was allowed to be conducted by the person you were critical of. It is certainly worth a letter to the editor and they certainly should not be surprised by it.

The third issue is the timing. This is not a problem, except as it implies a breach of protocol in points one or two above.

Once the work enters the public discourse then it is free game for comment. The person you are critical of did the right thing by delaying their publication until your work was published, and that is all you can really ask of anyone. If we are to have confidential peer review then there must be a point at which the restriction of confidentiality drops (in part because the reviewers are often the best qualified people to participate in the public discussion that follows the private review). You can see this especially with high-profile papers when they are published- all of the people who you think would normally be commenting are silent, then suddenly the paper is published and they all give a big sigh of relief and say, "OK, now I can talk about this."

As an author you might understandably want your work to have a "grace period" where it can be discussed freely without rebuttal, but as far as I know that's never been a thing. If you have to pick a point in time to drop confidentiality, then the public publication seems to be the best.

6
  • Hi David, I totally agree with point 1 and 3. Regarding your second point: so if I write a critique on paper A, then the authors of A should not be referees due to COI?
    – High GPA
    Jul 5 at 0:24
  • 1
    Conflict of interest policies usually only require disclosure, not recusal. It's up to the editors of the journal/conference to examine the COI and determine how it should be handled. Generally, if a paper was going to be a direct rebuttal and/or damaging to reputations, then the authors of A should not be reviewers. However, there are many fields where different groups will constantly "one-up" each other's work, and that's not a critique in a direct sense, nor is it a reason to exclude others from being reviewers.
    – David
    Jul 5 at 0:44
  • Many thanks for your teaching. So explicit criticism should not be reviewed by the original authors (usually). However, implicit "one-up" paper can be reviewed the original authors. Is my understanding mostly on track?
    – High GPA
    Jul 5 at 0:48
  • 1
    That is normal in my field, yes. Extending another author's work can be reviewed by that original author, and in fact they're probably one of the best people to do so.
    – David
    Jul 5 at 1:22
  • Why "it would seem that one of these two best practices was not adhered to"? I don't see a violation of confidentiality, as the preprint was issued after publication of the paper, and I don't understand what disclosure of conflict of interest was needed as, I assume, the paper clearly criticized the referee's work. I conclude that the editors were aware of the conflict of interest, furthermore, they wanted to know the opinion of the criticized person. Eventually, they accepted the paper, so they probably did not think the referee's objection warranted rejection.
    – akhmeteli
    Jul 5 at 11:44
3

I am not sure the reviewer's behavior is unethical. Technically, they did not violate confidentiality as their preprint was published after publication of your paper.

it seems like they were using our data/code

But again, they did not publish anything before your publication. After that, a preprint using your data/code looks OK (if your work product is properly cited). They probably were preparing their preprint before your paper was published, but I am not sure this is unethical, as the preprint was published after your paper was published.

it seems like they were ... withholding issues they spotted to prepare their own pre-print while reviewing my paper.

I assume the reviewer recommended rejection of your article. As others wrote, the reviewer could just make arguments that they believed were sufficient to warrant rejection.

The pre-print even includes a link that only appears in the draft that was used in the review process, not in the published paper.

If, as I assume, the link is accessible to general public, providing it does not violate confidentiality either.

In general, you criticized the reviewer's work, they published an objection as soon as they could. I would say, this is understandable.

0

This is an interesting question and I think it depends on which ethical framework you are operating under.

If you subscribe to a consequentialist ethics framework, then you would judge the ethics of this reviewer's actions by what the consequences are likely to be. If the reviewer did notice an error in your paper, but chose to withhold that information from you, then the foreseeable consequences include a paper possibly being published which contains one more error than it would otherwise. That's a bad outcome for the wider pursuit of knowledge (and for you as an individual), and the outcome is a foreseeable consequence of the reviewer's actions, so the reviewer has an ethical duty to inform you of the error via the review process.

On the other hand, if you subscribe to a deontological ethics framework, then you would judge the ethics of this reviewer's actions by what rules the reviewer was supposed to be following, or what the reviewer's responsibilities were. The purpose of a review is to make a recommendation to the editor regarding whether the paper is suitable for publication in that journal. If the reviewer honestly believes that the paper is unsalvageable, that fixing every error would not result in a paper suitable for publication, then their ethical obligation to make a sound recommendation using their best judgement does not entail mentioning every error they noticed in the paper. Keep in mind that although the review is sent to the author and the author is expected to respond to it, the purpose of the review is to help the editor make an informed decision about whether to accept or reject the paper.

Note: I've answered by assuming the premise of the question, which is that the reviewer did notice these particular errors before submitting their review to the editor. It's quite possible that they noticed later, in which case the question is more hypothetical. There is still a related question of whether they have an obligation to inform the editor about minor errors they discover after submitting their review; I'm not sure what the standard practices and norms are in such a situation, but even a consequentialist might say that if the reviewer sincerely believes that the editor will reject it based on their recommendation, then as far as the reviewer knows, informing them (and you) of additional minor errors would be inconsequential.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .