30

Kevin Buzzard's slides (PDF version) at a recent conference have really unsettled me.

In it, he mentions several examples in what one would imagine as very rigorous areas (e.g., algebraic geometry) were the top journals like Annals and Inventiones have published and never retracted papers which are now known to be wrong. He also mentions papers relying on unpublished results taken on trust that those who announced them indeed have a proof.

He writes about his own work:

[...] maybe some of my work in the p-adic Langlands philosophy relies on stuff that is wrong. Or maybe, perhaps less drastically, on stuff which is actually correct, but for which humanity does not actually have a complete proof. If our research is not reproducible, is it science? If my work in pure mathematics is neither useful nor 100 percent guaranteed to be correct, it is surely a waste of time.

He says that as a result, he switched to formalizing proofs completely, with e.g. Lean, which guarantees correctness, and thus reusability forever.

Just how widespread is the issue? Are most areas safe, or contaminated? For example, is there some way to track the not-retracted-but-wrong papers?

  • 21
    "Unscientific" is not the same as wrong. – Azor Ahai -- he him Jan 24 at 19:42
  • 5
    How could one possibly get to '100 percent guaranteed to be correct' without publishing things along the way? If you don't see what others are thinking and doing, and vice versa, you never get anywhere. – Jon Custer Jan 24 at 19:49
  • 14
    That said, I actually broadly agree with Buzzard's sense of urgency that "experts know this claim in that Annals paper is incorrect but it's OK because this other paper fixes it" is not a state of affairs we should be happy with. And also, my own area (functional analysis) is not pristine and immune from such things from time to time – Yemon Choi Jan 24 at 21:10
  • 9
    This question would be a better fit on mathoverflow, where I’m guessing you’ll get a variety of extremely well-informed and thoughtful answers by well-known mathematicians (possibly including Buzzard himself). I suggest migrating it. (+1 anyway, very interesting question and slides.) – Dan Romik Jan 24 at 22:51
  • 6
    @JonCuster No, but it's not a chicken and egg problem. It reduces verifying many complicated proofs to one simpler proof. See the FAQ about independent verifications of Lean: github.com/leanprover/lean/blob/master/doc/faq.md – Kimball Jan 25 at 1:28
25

I would say this is a common, but not fatal, issue. I think most mathematicians are aware of several papers in their field with serious errors/gaps without retractions/errata, and over time at least the wrong papers which are important get discovered. (e.g., see this MO post for temporary counterexamples---there aren't too many that went unnoticed for too long). There are also some grey areas where most people aren't sure if they can trust certain papers or not, and in certain areas/topics it's more of an issue than others.

The culture in mathematics is that retractions are typically for academic dishonesty, and incorrect papers either get corrected (or at least have their flaws exposed) in other papers or with errata.

What to do:

  • for an individual paper, check its review on MathSciNet, and check papers that reference that paper: if many people use it freely, it's probably okay, but if someone later found an issue they will often mention it in another paper. if the paper has a corrigendum, it will be linked in MathSciNet.

  • if you start reading the paper and for this or other reasons become suspicious, talk to experts---usually if a paper is important and there is a serious issue, experts become aware of the situation relatively quickly

  • check if the author has errata/comments on their webpage

  • some authors are more reliable than others, and they get a reputation for it; this is something you typically learn over time from your own research and talking to experts.

  • don't panic: if you happen to use a result that was wrong and don't realize it (or publish a paper yourself with a major error), that's okay. it happens from time to time and you can't check everything. you might discover the error later or someone might point it out. in any case, you are often able to then write another paper correcting the situation.

  • don't contribute to the problem: put errata for your own papers on your website and if there is a major error not corrected in a later paper, publish a corrigendum

  • (optional) keep track of known issues in a personal notebook along with their resolution. i do this because there are quite a number of papers in my area with errors/gaps and i have a bad memory. i contemplated making this public and letting other people contribute to this list, but i haven't so far mainly because i'm not sure i want to put in the effort to update it and make sure it is fair to all parties involved. however i have shared this list with some people individually.

| improve this answer | |
  • Thank you for the very interesting answer! – Archie Jan 25 at 6:50
  • As for your great list: what about creating a common website then? Something like math-errata.org where users' identity are checked and then they can update their own list. Since zentralblatt will become open access, it might even be something they'd consider hosting fiz-karlsruhe.de/en/nachricht/… The only point would be to protect against defamation of papers which are correct (feud between people). – Archie Jan 25 at 6:59
  • @Archie Well, it would be a lot of work to set this up and maintain it, especially if you want to moderate it for abuse/disputes. And MathSciNet already has a lot of this info through linked errata and linked reviews that many times point out when one paper corrects another. – Kimball Jan 25 at 13:19
  • 2
    I eventually update small errors that I’ve found or had pointed out in my old papers on the Arxiv. That’s another place you might look, and I find it more useful than having the paper and an errata in different locations. – Noah Snyder Jan 25 at 17:24
  • "if there is a major error not corrected in a later paper, publish a corrigendum" the problem is, in the specific case mentioned, this did not happen. Also: not everyone has a MathSciNet subscription. – David Roberts Feb 3 at 1:23
7

Journals don't retract scientific papers because they are now known to be wrong. If this were the case, they would have to retract a zillion papers from the past because, well, science evolves: new findings disprove old ones, old mistakes get corrected, and new ones are introduced.

And this happens in mathematics too.

This answer on MathOverflow makes a reference to the paper Errors and Corrections in Mathematics Literature which analyses the number of published corrections in mathematics literature, divided by field. The fraction of errata is less than 1%, but indeed this doesn't count all the undiscovered mistakes and those discovered which will never receive an errata.

| improve this answer | |
  • A somewhat related answer of mine: academia.stackexchange.com/a/48228/20058 – Massimo Ortolano Jan 24 at 19:45
  • 1
    Yes, thanks, but I'm asking how widespread the issue is in mathematics. I thought it would be marginal, and in particular top journals would never publish something incorrect : that means both the author and the referee(s) didn't catch the flaw. – Archie Jan 24 at 20:00
  • 2
    @Archie Have a look at the answer of mine linked above where I discuss a few examples, even though not from maths: but why do you think maths should be any different? Certain proofs are complex, difficult to follow and mathematicians are humans too (this is a strong assumption indeed :-) ): mistakes can slip. The point is that in a paper there is more than the conclusions ;-) – Massimo Ortolano Jan 24 at 20:06
  • 8
    “... and in particular top journals would never publish something incorrect.” Literally the first paper I ever spotted an error in was an Annals paper. And it wasn’t even a subtle error. It was a number field analog of calling the number 15 prime. – user109454 Jan 24 at 23:43
  • 4
    @BenLinowitz I am just curious, which paper are you referring to? – 2010 Kur Jan 25 at 5:00
2

Just how widespread is the issue?

This is not the be an exact answer to the question but a mention of some related famous anectodes and a mention of a research field dealing with the issue. This write-up by Voevodsky gives several examples of wrong proofs being belief to be true for many years. He eventually "pioneers" a research field called univalent foundations which aims to create a computer asisted proof checker (or create proofs that can be checked by computers).

| improve this answer | |
  • The article by Voevodsky is interesting, but maybe this should be a comment? – Kimball Jan 25 at 20:29
  • 1
    @Kimball I was suspecting the same. I thought I would have more to say but then realised the article itself was good enough. If a mod or another senior member susgests it I will turn it into a comment. – Boaty Mcboatface Jan 25 at 20:43

Not the answer you're looking for? Browse other questions tagged or ask your own question.