3

I submitted a paper in a major machine learning conference recently. The paper submission deadline passed ~2 weeks ago and the appendix submission deadline passed ~1 week ago. I just realized an issue in my testing pipeline that made the benchmark numbers incorrect. I reran the benchmarks and the numbers don't change the results (in the same "category"). What do I do (withdraw or request amendment)? I would not want the paper to get published with incorrect numbers (out of academic integrity), but it realistically should also not change much review wise.

2
  • 2
    Do you know whether for this conference it is standard that they ask for revisions? In that case you have a chance to correct things without trouble. In many places even if they accept the paper without revisions you'd have a chance to do corrections before final publication. Given that main results are unchanged, it shouldn't be a problem to correct this if at any point you have a chance to make changes, although I can't know whether at this conference you'll get such a chance (more likely than not I'd say, but what do I know... maybe ask somebody with experience with this conference.) Oct 16, 2023 at 0:07
  • @ChristianHennig - I'm not entirely certain. I can always email the chair. I imagine it would not be wise to just update during the de-anonomization update without a note
    – mb1
    Oct 16, 2023 at 1:02

2 Answers 2

3

It sounds like you are unaware that the paper, if accepted, will not be published in the verbatim form that you submitted to the conference, but there will be a camera ready version, where you are supposed to incorporate reviewer comments, and this will be the final version that will go into conference proceedings.

If these changes do not change the results and conclusions, then you can update the results with the latest ones and submit it as the camera ready version.

If the updated results do change conclusions, then the paper should be withdrawn as in most conferences you cannot update the paper after it has been submitted and the deadline passed.

As an example, for the Neural Processing Systems Conference (NeurIPS), this conference has a rebuttal process where authors can add new results in their rebuttal,in their 2023 FAQ they clearly specify this:

Can the rebuttal include new results? Yes, however your original submission will serve as the basis for the reviewers' (and ACs') acceptance recommendations. The rebuttals should serve only to clarify the reviewers' and ACs' questions during the discussion period.

And for the International Conference in Learning Representations (ICLR), you can update the paper during rebuttal, as long as you are clear to the reviewers of what has changed. This is very clear in the FAQ:

Q. Can we change the title of a paper during the rebuttal? Yes, you can change the title, abstract, and the paper’s content, including supplementary materials. But make sure any modifications are clearly communicated to the reviewers and the area chair, so that they can efficiently review the modified version of your paper. The set of authors cannot be changed, but the order can be changed.

4
  • I was aware, but my impression was that camera ready was for cleaning up (formatting, citations, acknowledgements, etc.) rather than changes to the content (small as they may be) of the paper itself
    – mb1
    Oct 18, 2023 at 3:03
  • @mb1 you’re not wrong, that’s precisely what camera-ready versions are suppose to be. This answer asks you to abuse the review process and commit an ethical violation. Will you be caught? Probably not. Is it the right thing to do? Absolutely not.
    – Spark
    Oct 18, 2023 at 9:08
  • @Spark Sorry, what? You seem to be unaware of standards in Machine Learning conferences, this is not an ethical violation, people are allowed to add results and changes to the final version, please put evidence to your statements, in particular for Machine Learning conferences.
    – Dr. Snoopy
    Oct 18, 2023 at 9:18
  • @mb1 Your impression is actually incorrect for ML conferences.
    – Dr. Snoopy
    Oct 18, 2023 at 9:30
-1

The ethical thing to do is to withdraw the paper. We are scientists, and our goal is to tell the truth about the world as we see it. If you made a mistake, minor as it may be - own it and fess up. Here’s a few other reasons:

  1. It’s a slippery slope: if you let yourself go this time, you may make greater allowances the next time.
  2. It’s better if you own up than if a reviewer finds out/the numbers don’t make sense. Someone might ask to see your code for reproducibility. What will you tell them?
  3. One discovered error usually implies that other errors exist. Can you say for sure they don’t?
  4. ML is suffering from a serious reproducibility crisis, don’t contribute to it- be a force for good!
7
  • 3
    I think it makes more sense to just update the paper rather than withdraw it (if allowed). Surely there is a way to contact the chair and let them decide.
    – sErISaNo
    Oct 16, 2023 at 4:56
  • Most top ML conferences don’t operate this way. Once you submit there’s no real option for revision (say if it’s NeuIPS ICML ICLR etc)
    – Spark
    Oct 16, 2023 at 13:15
  • Surely you still get to re-upload a camera ready version if the paper gets accepted
    – penelope
    Oct 17, 2023 at 10:26
  • Yes, but still, the review itself was on wrong results. It’s not the right thing to do.
    – Spark
    Oct 17, 2023 at 12:34
  • 2
    @Spark In ICLR you can update the whole paper in rebuttal, what you said is plain wrong: iclr.cc/Conferences/2024/AuthorGuide "Q. Can we change the title of a paper during the rebuttal? Yes, you can change the title, abstract, and the paper’s content, including supplementary materials. But make sure any modifications are clearly communicated to the reviewers and the area chair, so that they can efficiently review the modified version of your paper. The set of authors cannot be changed, but the order can be changed."
    – Dr. Snoopy
    Oct 18, 2023 at 9:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .