8

I am reading a paper published in a SCI-core journal.

After a very careful and tedious evaluation, I have come to the conclusion that the authors have made serious mistakes.

Should I write to the editor of the journal about it so that other people do not waste their time on this paper and may learn a thing or two about it? How can we do that?

Improve this question
3
  • 5
    Have you read the Wikipedia article? en.wikipedia.org/wiki/Letter_to_the_editor
    – Roland
    Jun 12, 2020 at 9:23
  • Nops I did not read it earlier. Thanks for notifying me. If you share it as an answer, I will accept it
    – SJa
    Jun 12, 2020 at 9:38
  • Most wrong papers will not ever be used for anything. Is there reason to think other people will make use of this paper?
    – Anonymous Physicist
    Jun 13, 2020 at 6:50

5 Answers 5

Reset to default
43

Assuming you are have carefully checked that there really are serious errors, the best course of action would be to contact the authors of the paper first. Maybe they can explain subtle details, not clearly described in the paper, that explain everything. Of course it is important to be nice: It is very well possible that everything appeared to be correct with the knowledge at the time the paper was published. Or that it is you who misinterpreted the methods (possibly because the paper was a little vague). And even if there are errors: nobody is perfect. So please be nice.

It is possible that the authors wish to collaborate with you to correct the errors, leading to a shared publication: a win-win situation. I have seen this happen more than once.

If this does not work for some reason you can consider contacting the editor, who may or may not be interested in correcting the errors. That will be up to the editor.

Depending on your specific case, it may also be possible to do some experiments yourself and publish your findings, referencing the paper with the errors and correcting them.

However, if the errors are minor, the result is "unimportant", or you are not too sure that there really is a mistake that needs correcting, you could also decide to simply ignore the paper. There are plenty of bad and/or unimportant papers out there and frankly, they do not deserve the attention even if it is to correct mistakes.

Note: I have partly copied from my own answer to a different, related question.

Improve this answer
3
  • 1
    You begin your (good) answer by assuming 100% certainty. I suppose that can actually happen once in a while. In practice, and especially in my own experience, one can only be "mostly sure." Most of the times I've been mostly sure of an error there actually was an error of some kind, but I think it would be more realistic to proceed from the premise of less than complete certainty. If you have a serious concern that there is a serious error but are not completely sure, whether it is appropriate to ignore it really depends on the situation (I think).
    – Pete L. Clark
    Jun 13, 2020 at 17:22
  • You are right. I did not mean the 100% so literally but just wanted to emphasise that OP should check, and double check that there really is an error before taking action. I have edited my answer.
    – Louic
    Jun 13, 2020 at 20:48
  • 2
    If Sjaffry follows this advice, it would be interesting to know the results achieved and time spent on this course of action. Hopefully, see you in a few months!
    – Sylvain Ribault
    Jun 15, 2020 at 17:19
7

To elaborate on @anpami's sensible answer: in most fields of science we have a rotten publishing system which rarely corrects errors. Depending on the field, you can expect 20-80% of published papers to be false, only a small fraction of which are ever corrected. In a publish or perish world, authors and editors have strong incentives not to correct mistakes.

Correcting mistakes via the journal that publishes them is very slow and time-consuming: by the time a correction appears (if ever) most of the damage will be done. So you should look for other routes in order to quickly broadcast your opinon on that article.

The venue to do that depends on your field. PubPeer would be a possibility. In physics or mathematics you could write a comment as an Arxiv preprint. If the paper in question is on Biorxiv, you can directly write your comment there.

Improve this answer
10
  • 3
    I agree that anpami and you correctly identify an existing problem in the publishing system, but the question was "should I write to the editor". The choice then is (1) No, and be part of the problem, or (2) Yes, and be part of the solution. I am not against leaving comments on a website, but it is much more polite (and useful) to contact the authors of the paper first, and then contact the journal together.
    – Louic
    Jun 12, 2020 at 19:33
  • 1
    @Louic : I do not know what the solution is, but trying to correct wrong papers one by one via journals is hopeless. (Too much work by orders of magnitude.) You do not reform a rotten system by following its rules, but by exerting external pressure.
    – Sylvain Ribault
    Jun 13, 2020 at 8:25
  • 1
    @ZeroTheHero: Then have a look at en.wikipedia.org/wiki/Replication_crisis . In some fields, only a minority of studies can be replicated, therefore the rest are wrong. And bear in mind that a replicable study can still be wrong in many other ways.
    – Sylvain Ribault
    Jun 13, 2020 at 18:35
  • 1
    @ Mazurka Fahr: Thank you for sharing your experience. This reminds me of en.wikipedia.org/wiki/Elisabeth_Bik , whose occupation is to detect photo manipulations in articles. As far as I understand, only a fraction of detected manipulations lead to corrections in journals.
    – Sylvain Ribault
    Jun 13, 2020 at 18:54
  • 5
    @MazurkaFahr it is NOT the job of the editor to verify the results: if anything it is unlikely to be technically possible. As an extreme example, the editors of Phys.Rev. Letters cannot possibly reproduce results from LIGO or CERN. Their job, however, is to make sure that the manuscript is properly vetted and assess the validity of any objection pre or post-publication. Certainly they should make efforts to correct errors once these are brought to their attention, possibly by contacting again referees or assessors in the light of new information.
    – ZeroTheHero
    Jun 13, 2020 at 21:29
2

Editors may be reluctant to publish your letter. In that case, PubPeer - which offers the possibility to write publicly available (critical) comments on research papers - may offer a good alternative.

Improve this answer
4
  • 2
    I very much disagree with this answer. It the responsibility of a scientist to correct mistakes, regardless of who made them and why. If the errors are serious, something should be done about it to prevent others to continue building on this erroneous information. See also my related answer here: academia.stackexchange.com/questions/98833/…
    – Louic
    Jun 12, 2020 at 9:59
  • 1
    You are depicting how the ideal situation should be, Louic. But in reality, editors are not always eager to publish critical comments or even to issue retractions. That's why PubPeer exists (and other similar platforms, e.g. Retraction Watch's blog).
    – anpami
    Jun 12, 2020 at 10:01
  • 1
    That does not mean he should not write to the editor, although it would be preferable to contact the authors of the paper first. They may be very much interested in correcting the errors.
    – Louic
    Jun 12, 2020 at 10:02
  • @anpami Do you have empirical evidence to your claim that editors are not always eager?
    – Wolfgang Bangerth
    Jun 12, 2020 at 15:15
1

I think the best thing to do is contact the authors. I expect that you'd want to be contacted if there was a potential issue with one of your papers! If they don't get back to you within about a week, it might make sense to contact the editor.

If you contact the authors, you may find that situation is not as bad as it seems:

  1. Perhaps the paper's conclusions are correct, it's just that an implementation detail in the paper is wrong.
  2. Maybe it just doesn't generalize to your use case e.g. due to differences in data preprocessing.

Remember, most papers are reviewed by the authors and their colleagues many times, in addition to being reviewed by editors and external reviewers. As a result, I think it's important to be respectful and report your findings in a clear and constructive way. If your findings have not been reviewed yet, you're asking the authors of the original paper to review your findings.

Finally, it may make sense to publish a paper or corrigendum on your conflicting results. This might be a substantial contribution to your field. You could also post a preprint on your preliminary results, e.g. to the arXiv, to raise awareness.

Improve this answer
1

The first thing is to contact the authors and politely ask for clarifications. It could be a “simple” oversight at any point in the process.

If you are not satisfied with the answer, then the next step would be prepare a Comment describing the issue and submit this to the same journal for publication. Presumably if your objections are prima faciae valid the editor would then contact the authors of the original paper for a reply, and the Comment plus rebuttal would then be published side by side if this clarifies the published paper.

The last option would be to publish this comment on arXiv or any such platforms. In particular if the original erroneous manuscript was published on arXiv then sending the comment there would be the natural thing to do. Doing this escalates the dispute to the public level and is often seen as agressive, especially if you have not tried to reach out to the authors first.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .