50

I am a final year undergraduate student and had worked on a very fundamental project(there is no available literature on this field presently) during my sophomore year. The results obtained were quite interesting but there was no available literature to back up my results. My supervisor suggested me to submit it to a conference and get reviews on the work.

I did the same and when I submitted the final paper, I got a strong accept with quite positive feedback. I presented the paper at the conference and was also appreciated by my conference chair who said that it was an interesting finding. However, while working on the journal version recently, I discovered a severe error in my paper which completely invalidated the claims I had made in my work.

The conference was highly reputed and delegates from all over the world were present. I feel really ashamed of presenting the wrong work at such a big stage. I discussed this with my supervisor and he told me that we will decline for publication by stating the reason when they will send the copyright form for inclusion in the digital proceedings. This gave me a sense of relief.

However, I cannot stop thinking about having presented the wrong work and seems like I should not have gone for it in the first place. Please suggest me what should I do. I have a deep interest in research but I am in constant fear that my habit of premature celebration will land me into trouble someday.

  • 30
    Hey, don't beat yourself up. Near as I can tell, you are ready to join my industry. Admitting your mistake as soon as you find it is the crucial thing. – puppetsock Jan 21 at 16:42
  • 11
    Relevant xkcd xkcd.com/2239 – yesennes Jan 21 at 18:27
  • 14
    It sounds like a lot of people missed this mistake, not just you. A short note on why this mistake was so easily missed might help others avoid it in the future. This isn't a necessary step to take, but if it's achievable, it could be very valuable. – Dancrumb Jan 21 at 20:15
  • 2
    There's another aspect to this. A coworker wrote an analysis program for thermal hydraulics in nuclear reactors. In the first version, the coolant came out of the core at supersonic speeds, and at a temperature of negative 30°C. The t-shirt for the project was great! Later he fixed his error. Still later he became head of the department. – puppetsock Jan 22 at 21:33
  • 2
92

Let's see:

  • You wrote a paper of sufficient writing quality that it was chosen for presentation at a conference and publication.
  • None of the peer reviewers noticed anything wrong with it.
  • None of the people in the audience questioned it.
  • Your supervisor saw nothing wrong with it.
  • You gave an excellent presentation.
  • You found a flaw in a paper that had already been accepted for publication.
  • You had the integrity to withdraw your paper from publication.

That's a long list of things to put on your "plus" list. Most undergrads don't achieve even one of them. None of them should be seen as a negative.


Be warned that you'll become more jaded as you progress.

I remember as a grad student being asked by my supervisor to review a paper that had been sent for him to review. It was well written and obviously the result of a lot of hard work, but I found a fundamental oversight that would have allowed the entire problem to be reduced to something very trivial.

We told the publishers about this, and they wrote back requesting that we recommend it for publication anyway. My supervisor said that the author could publish another paper later that would re-analyze the problem and present it differently.

That's one (of many) reasons I eventually didn't bother finishing my own PhD.

  • Thank you very much, Ray. That really helped – user118612 Jan 21 at 16:32
  • 7
    @user118612 I built a physical device for my thesis. It didn't work. In fact, my original theory was so far off it couldn't reasonably be made to work. I took numerous measurements, explained in detail the reasons it didn't work, and got an A. A negative result is still a result. – Cristobol Polychronopolis Jan 21 at 19:44
  • 4
    ELI5 Ray's second-to-last paragraph. They published anyway so they could get a second publication out? That would basically say "remember that paper last month on X? Well, turns out we improved our method!" – Mindwin Jan 21 at 21:20
  • 5
    @CristobolPolychronopolis there is no such thing as a “negative result” from a well conducted study, and its unhelpful to labels results which are not what was hoped as negative. It makes people devalue very valuable study outputs. Knowing something doesn’t work is often more useful than knowing it does. – rhialto Jan 22 at 0:02
  • 3
    @rhialto I meant negative in that the hypothesis was disproved. I agree that the knowledge is just as valuable either way. Did you read my comment past the second word? Has "negative" made the politically incorrect list now? I hope my doctor feels free to continue using it. – Cristobol Polychronopolis Jan 22 at 13:28
36

Relax. You're not the first person to make a mistake, and people are not likely to know that you had been celebrating (besides, even if you were ... so what?). The fact that you found the error yourself is furthermore a good sign, since it means you're taking your work seriously and subjecting it to the scrutiny it deserves.

Take a look at this. The first direct evidence for dark energy goes back to the late 1990s, when two groups claimed to have discovered it. This was a big deal since dark energy supposedly makes up ~70% of the energy content of our universe. After follow-up work confirmed it, the 2011 Nobel Prize in Physics was awarded to the discoverers.

And then in 2019, we get this paper:

Thus the cosmic acceleration deduced from supernovae may be an artefact of our being non-Copernican observers, rather than evidence for a dominant component of “dark energy” in the Universe.

In other words: the paper claims the discovery was wrong. Let's say that again: this recently-published paper claims that the 2011 Nobel Prize in Physics was awarded incorrectly.

Imagine you are one of the three 2011 Nobel laureates in physics who's been celebrating for 8 years, and this result turns out to be robust. Is your reaction:

  • Yay! Our knowledge of the universe has advanced yet again! Or is it:
  • Oh my God, people are going to look at me and think I'm an embarrassing failure. I've been celebrating for twenty years, given plenty of talks discussing my results, even won a Nobel Prize, and my work is wrong. Panic!

I think most people will think the second reaction is pretty silly. If you think so too, then listen to your supervisor, who's a much more experienced academic than you and should know what to do. Perhaps there's something you can salvage from the work, which might make it less interesting but still a solid result, for example. Or perhaps the method you used can be adapted to a different problem. Either way, you've learned something. If you stay in research, you'll be meeting this kind of issue often, so don't worry too much about it.

  • OT how normal it is that the pdf you have linked is for free download? It is a sign of its special importance? Normally journals do so with breaking outcomes and results. Of course this might be – Alchimista Jan 21 at 8:15
  • Thank you very much Allure. I hope withdrawing would be the right option. In case I don't submit the copyright form, will that affect my association with the conference organizers? – user118612 Jan 21 at 14:39
  • A rebuttal of the paper you mention is found here: arxiv.org/abs/1912.02191. However, I agree with your general sentiment. Science is often wrong. Finding mistakes is how we progress. – Teun Jan 23 at 10:04
  • 1
    A 2018 Nobel Prize winner recently had to retract a paper as well: bbc.com/news/world-us-canada-50989423 Good company :) – smcs Jan 23 at 10:37
12

There is one other thing you can consider doing, after getting agreement from the conference committee.

You can put a note in the paper for publication, noting that you have found a serious error and explaining the nature of the error. Don't bother with apologies and such, but just point out where the paper is wrong.

This assumes, of course, that you don't have a correction. If you do, then a corrected version for publication would be better.

As your advisor notes, withdrawing it from publication is also a good plan, but a notice that you can honestly evaluate a failed result is valuable to those who have already seen the work (at the conference) and might want to see the outcome.

But yes, relax. Any errors here are shared with a lot of people.

  • The second option seems better to me since I am presently in my final semester and would be graduating soon. The work would rather require a detailed analysis and I have also switched my field(I am carrying over the project which I did during my internship). I feel that I should withdraw my paper since I have the option, work on the problem again from scratch(if time permits) and maybe submit the corrected work to some other conference, what do you suggest? Also, will not submitting the copyright form affect my association with the conference organizers? – user118612 Jan 21 at 14:34
  • 8
    You have a decent plan. But you need to be more proactive with the committee. At least let them know why you believe it isn't appropriate to publish it in the proceedings and that you wish to withdraw it. They may have suggestions. But just silently withholding copyright won't win you any favors for the future. Make it explicit. – Buffy Jan 21 at 14:41
7

Your list of accomplishments is still very impressive, and I would moreover add that this is extremely common, even for very established researchers. I don't know your field, but in computer science my empirical belief is that the majority of papers contain mistakes; usually minor and hopefully simple fixes, sometimes more major fixes that don't invalidate the whole paper, and occasionally something major that invalidates the paper. Such mistakes are not a serious cause for concern as long as you acknowledge them professionally, which you have already done.

For instance, in theoretical computer science, Laci Babai raised a lot of buzz for posting a proof of a major open result, then later retracting it saying the result was not true, and then later reinstating the claim and saying he had a fix. All this was posted on his personal webpage and went out as news to the entire computer science community; you can imagine the embarrassment he must have faced when having to announce that this major result everyone was excited about was wrong! Yet he still announced it, and he is still well regarded by the community.

People understand that sometimes results can be wrong for subtle reasons -- that's the whole point of doing research, to find errors, fix them, iterate, and progress. If everything were already known and certain, there would be no work left to do!

3

I agree with the other answers, but think that it's worth considering what it's worth to have your contribution out there for others to benefit from--the whole point is to make each other smarter!

As perhaps an extreme example, if this were medicine, and your imperfect paper could jump start others in saving lives, then surely it would be moral to make the world better by publishing despite the flaws. Presumably your case isn't as extreme, but if the next opportunity to share your value with the world isn't for another year or so, then getting people moving in the right direction is still probably better than withholding valuable information or ideas.

  • 2
    Medicine seems like a strange example to use, given that if there were a fundamental flaw in an important medical finding that had impact on people's lives, it would be especially important to redact the paper and publicize the flaw immediately. – 6005 Jan 22 at 21:23
  • Agreed. I was focusing on the academic research part of the process, vs. the procedure development or drug production part. CRISPR, for example. – Eliot Gillum Jan 25 at 2:12
3

Others told you already that it is not uncommon to have mistakes or errors in papers.

I go a step further: consider that scientific work means being right at the edge of what is currently known. Honest mistakes will happen every so often, and maybe more often in science as a consequence of venturing into the unknown*, and without any particular fault of the one who makes them. In the sense that good scientific work may still fool you at some point and only later on that will be realized.

* and the additional "complication" that AFAIK there is no known procedure to ensure that all potentially important influencing factors have been considered. It's a genuine professional art to not miss important influencing factors.


So the task at hand is to deal with the mistake and learn from it.

That being said, I've once been in a similar situation. Just that I found my mistake (which was of the embarrassing sort that could and should have been avoided; and could have been caught early on had I formulated better unit tests than I had - but in hindsight it's easy to see how the testing could have been better.) With the difference that I found my mistake after manuscript submission.

The procedure then was to retract the manuscript with a letter explaining that a mistake had been found so that we please ask to retract the manuscript. IIRC the editor responded that if we hand in a corrected version until $deadline it would be considered a resubmission - otherwise (later) we'd be required to submit as a new paper. That was all, and I can tell you that it's not particularly painful.

Also, as a reviewer I once received an email just a few days after the manuscript came (so long before the review deadline) that my review was not needed any more. No reason was given, so I don't know whether the manuscript was retracted or whether there was another reason.

Lessons learned:

  • gained experience in formulating unit tests.
  • retracting a manuscript because a mistake has been discovered is more or less a standard procedure and a proper step for dealing with a mistake.

My recommendation to OP:

  • correct the mistake and then submit the paper. If it's too late for the proceedings, submit to a journal.
  • if your presentation slides/poster is publicly available, put a warning slide/notice on it that a mistake was discovered and later on that a corrected version was submitted: $link-to-paper.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.