Dealing with the inevitable presence of mistakes in paper submissions

Question

When writing a paper, there are inevitably mistakes. Many of these can be caught by careful revision, and the more time spend revising, the more mistakes which will be caught. However, clearly a draft should be submitted at some point.

How should one determine this cutoff point? How much does it vary by field, individual, etc?

As a concrete example, I recently submitted a paper to a big ML conference where one of our results was off by a factor of two. While we fortunately caught this error before the camera ready version, the experience has caused me to question whether I am spending sufficient time on revising my work before submission. I have heard that such errors repeatedly occurring in published work can seriously damage your reputation in math.

On the other hand, I wonder if the standard for errors like this differ across disciplines. For instance, if we had submitted the paper to a journal in applied math, then I believe such an error would have been caught by the referees since the review process there is more fine grained. Thus, it also seems possible that fields in CS, where conferences with short review processes (relative to math journals) are standard, have a lower threshold on what what kind of errors are acceptable to the community.

I've already discussed this with my advisor. However, I am also interested in hearing the thoughts of others on the issue.

---

Edit: Thanks everyone for the responses! I'd also be interested in the second part of the question about how these standards vary across disciplines, individuals, etc.

Answer 1

Ideally, the desired error number is zero. But as you observe, it is difficult to achieve. One problem is that the author of a paper is probably the least able to proofread it. (Maybe not literally "least", but I hope you get the point.) The reason is that when reading your own work what you "see" on the page is too often what you thought your wrote, not what you did write. Your brain glosses over errors.

What you need to stamp out errors is a set of fresh eyes of people with enough background to find any errors. They aren't influenced by what you were thinking when you wrote the paper. This is one of the main benefits of the reviewer system, actually.

An important author in CS (Don Knuth) has the practice of funding bounties for errors found in his book. He will send a check to the first person to catch any error. (He is so renowned, however, that few cash the checks anymore.)

So, you do what you can, being as careful as you can, for as long as is reasonable. But get some new "eyes" on your work.

---

Note also that some important errors in important papers have gone uncaught for decades. I once found an error in a paper by a renowned mathematician about 40 years after the paper was published. I don't claim to be the first to notice, of course.

I'm working on a long term project and have, after a few versions, still seen some incredibly bone-headed errors. The biggest thing holding me back is the lack of those "other" eyes.

Answer 2

In addition to typos-versus-flawed-proofs, there is another distinction worth making, I think. Namely, some proofs really are (perhaps) delicate computations (or other intensely symbolic lines of reasoning), and thus their persuasiveness and believability is already somewhat fragile. For example, here typos are terrible.

In some contrast, some written arguments are more "narratives" of a procedure/process that could be carried out, with many details visibly dictated by "the plan". If/when the details are determined in a more top-down fashion (or can be made to seem so), the argument is much less sensitive to typos (or even larger blunders...)

I'm not claiming that there's a dichotomy of innate nature of arguments. Rather, I'm advocating more-narrative approaches to what might otherwise invite fairly fragile, long computations without a good accompanying narrative. More-narrative approaches can be more robust, and less sensitive to typos and other errors.

Answer 3

Here is my approach (pure math):

• I consider a paper ready for submission when I personally am completely convinced of it's correctness. If I only think, "it's probably okay", that's not good enough. That said, I do put more effort into verifying central statements than tangential asides.

• When possible, I try to cross-check my results in different ways (do they contradict other approaches or heursitics) and compute examples. For me, computing examples and comparing with related results in the literature is very important for finding errors and convincing myself my finalized version is correct.

• Explain the work to others, and ask others for feedback.

• After I finish writing the paper, I put it aside for a couple of weeks, and then reread it as carefully as I have the energy to. If there were significant changes, repeat this. Do this again after referee reports.

This doesn't mean all of my papers are error free, but as far as I know there are no major errors in the final versions of my papers.

Answer 4

Depends on mistakes. If these are crucial mistakes in arguments, you should correct them before submitting the paper. It is doable. Misprints, on the other hand, are not that important and you do not need to catch them all. So to submit your paper you need to be absolutely sure that the arguments are correct. Recently I edited a paper submitted to "my" journal. The referee found that all proofs are correct. Then I found lots of small misprints and gave the authors a few weeks to proofread the text. When they do that, I intend to recommend the paper for acceptance.

Answer 5

Assume the perspective of a Reviewer.

As a Reviewer, I will never reject a paper because of "mere" typos unless:

• they make the paper unreadable. But this happens when the author(s) spend no time in spellchecking, or have very little English writing expertise (and both of these are different problems)

• they are still widely present after the first round of review. I can tolerate some typos in the first submission, but my acceptance level decreases for subsequent rounds, especially if I made it clear that the paper required revisions in the English text (but this is also a different circumstance than what you are likely referring to)

• the typo affects a crucial claim. These are the most crucial offenders, because these "mistakes" undermine the credibility of the paper and can be the ground for rejection. I once rejected a paper because the authors stated "X" as the main contribution of the paper in the Introduction, and then stated "not X" as the main contribution of the paper in Section 2; the rest of the text did not allow me to determine which was right, and most of the remaining paper was too complex for my expertise, so I suggested a rejection. Note that this was explicitly due to the usage of the word "not", which can single handedly flip the table around.

• the typo affects Proofs. This is self explanatory and is strictly related to the previous point, but it demanded a dedicated entry.

• the typo affects the Results (numbers, tables, figures). The result section is arguably (depending on the field) very important and usually the source of many significant mistakes - which may or may not be considered as typos. Consider a 2d plot where you have two horizontal lines, a red at y=3 and a blue at y=6: if in the text you say that "the blue line is clearly lower" that is a big problem. Was the mistake due to (i) the incorrect use of the "black/red" color, (ii) incorrect use of the "lower/higher" term, or (iii) maybe it was a problem in the plot? As a Reviewer I do not know, and hence I can use this issue to reject the paper.

In summary, my recommendation is to try to point out the key areas in your paper that are likely to be read by Reviewers and focus on having those completely free of mistakes. Perfection is difficult to guarantee, so at least focus on the big picture.

As a final suggestion, make the Abstract error free: abstract with typos are a red flag.

Answer 6

Perspective of a recent PhD student here:

You learn to make papers "better" by writing papers. And by "better" here I mean useful to your audience. The goal of academic writing is not to create perfect piece of text that will stay valid forever, but to help others and to move the scientific discussion forward. You can do it even when you have mistakes and typos.

By writing and submitting more work you will improve your work process (for me it was learning to slow down and letting papers "marinate" for a few weeks without me working on them). You will also see your mistakes published, which will remind you to try harder next time.

However, typos (not critical logic/math mistakes) and other blemishes are not disqualifiers for an academic piece to be useful.