26

How does a math journal react to a crackpot math research paper? In mathematics (especially in Number Theory) there are many meaningless papers on the web, and possibly a ton of meaningless submissions. Also, how does a journal react when a proof of a major unsolved problems, such as the 'Riemann Hypothesis', is submitted?

22
  • 19
    As an editor, I'd immediately reject these papers. If the paper made enough sense to have an identifiable error (or small set of errors), I might point out the error(s); this is often but not always useless. If the paper was just nonsense, I'd just reject it. (One exception: A journal promised blind refereeing, so I felt obligated to ask for a report from a referee who hadn't seen the author's name. I shouldn't have felt that way.) – Andreas Blass Jan 31 at 18:40
  • 22
    @MathiasMüller: because the limit on time that the journal has to vet papers is lower than the corresponding limit on time that crackpots have to generate them. When faced with an argument that has no mathematical merit, it doesn't necessarily take careful vetting to see this. The tiny risk of missing a paper that has some merit, but fails to make it obvious, is justified by the value of redirecting resources towards submissions with obvious merit. – Steve Jessop Feb 1 at 9:24
  • 12
    But I say "almost" certainly: an elementary proof in a single paper is conceivable, just not likely enough to justify spending time on a potentially-unlimited quantity of utter nonsense. The fact that mathematical journals exist does not entitle me to submit 26MB of randomly-genereated ASCII text and expect a journal editor to read it carefully just in case it contains a proof of the Goldbach Conjecture. – Steve Jessop Feb 1 at 9:30
  • 17
    @MathiasMüller As the author of several >100 pages papers, I disagree :) (I swear we tried to make them shorter) – Denis Nardin Feb 1 at 16:35
  • 6
    Please don’t write answers in comments. It bypasses our quality measures by not having voting (both up and down) available on comments, as well as having other problems detailed on meta. Andreas, Steve -- please consider converting your comments into answers; they seem like good answers that would be well received. – cag51 Feb 1 at 19:22
41

Here are the basic mechanisms that I know of, having only limited experience on an editorial board.

The first is a classic desk rejection. This is where the editor who makes the initial assessment, just says no, without asking for referees. This has to happen, or one gets flooded with AI generated nonsense as a joke.

Some journals insist that you list a few potential referees. I believe that one of the reasons they do this is to see if you have interactions (even just closely reading their papers) with mathematicians. If the author of a number theory paper list as potential referees three Fields medalists from three different areas of mathematics, probably they have no idea what they are doing. Makes a desk reject easier.

The second is the editor asks for a quick opinion from a mathematician in vaguely the correct area, asking if the paper is worth refereeing. You can tell if you get this type of rejection sometimes as the editor says after a few weeks something like "the feedback I have gotten are that this is out of scope for our journal." So I have heard. From a friend.

Finally there is just sending it to one or more reviewers. If the editor needs one referee, asks four in succession and they all say no, then the editor is likely to reject the paper.

There are problems with these methods. Some innovative work is hard to classify, and a paper gets rejected because the editor asks all the wrong people. I had a paper go to a topic editor, go out for review, then get assigned to another topic editor, out for several reviews. It finally landed on the correct reviewer, but this could easily have gotten rejected.

12
  • 13
    Some journals insist that you list a few potential referees. I believe that one of the reasons they do this is to see if you have interactions with mathematicians I used to be a physicist and had my fair share of lunatics coming to me with new theories. I had two cases of people who came with an innovative approach in a rather niche area and they did not know anyone in the field. Their papers ended up being published. So I hope that this requirement can be waived (when correctly explained) because it may, in these rare cases, be a show stopper for some. – WoJ Feb 1 at 8:22
  • 16
    Judging a paper based on the suggested referees, rather than the content of the paper, is outright corrupt. Do not do it. – Anonymous Physicist Feb 1 at 8:48
  • 35
    @AnonymousPhysicist Is corrupt really the right word? What does an editor personally gain from judging a paper by its suggested referees? I agree a paper should be judged by its content, but suggested referees still seems like a useful data point to contextualize the paper. If an author cannot describe how their work fits into the field at large, or name anyone who has worked on similar problems, I'd certainly look at their manuscript with a more critical eye, since the author has evidently not judged themselves in the context of the relevant body of work. – Nuclear Hoagie Feb 1 at 15:31
  • 9
    @WoJ Do you mean that they did not know anyone personally in the field, or that they did not know the name of anyone working in the field? In the latter case, even assuming that someone could make innovative research without being familiar with the past work, it can be easily fixed with a cursory literature search... – Denis Nardin Feb 1 at 16:29
  • 7
    @DenisNardin: yes, I meant personally. In retrospect, I see that the intent of the request is to suggest appropriate reviewers, rather than ones that one knows personally. This indeed makes sense to check if some has an idea of who works in the field. – WoJ Feb 1 at 16:42
26

A famous number theorist once told me he receives approximately one submission a week claiming to prove the Riemann Hypothesis at the top journal where he is an editor. Thus the need to conscientiously and in good faith evaluate such submissions is a real drain on his time and the time of other experts in the community. It’s not an acceptable solution to reject them all without even looking at them, and neither is it acceptable to handle them all using the “usual” process.

Fortunately, if I remember correctly what he told me, he can rule out all but an extremely small percentage of those submissions as being obviously invalid (where “obviously” means he doesn’t have to spend more than 5 minutes looking at them, and can then desk-reject them with an easy conscience and even sometimes provide a bit of feedback to the authors) by applying a simple set of heuristics he developed over time. They go something like:

  • the paper doesn’t use standard notation or terminology, or otherwise show some obvious misunderstanding of the problem or related body of knowledge

  • the paper uses only properties of the Riemann zeta function that are shared by a larger class of functions, for some of which the analogue of the Riemann hypothesis is known to be false

  • the paper proves something that’s actually much stronger than the Riemann hypothesis, and known to be false

[etc - there are probably more subtle rules of thumb that are still considered by him very trustworthy given his expertise in the subject]

I believe other experts who are regularly asked to evaluate purported proofs of famous open problems have developed similar heuristics. See for example Scott Aaronson’s “Ten Signs a Claimed Mathematical Breakthrough is Wrong”.

The bottom line is that it is actually quite difficult to come up with a crackpot paper that doesn’t instantly fail a set of obvious sanity checks that experts will know about but the crackpots (or honest amateur mathematicians who are not complete crackpots but simply lack training in the area) don’t. The only people who actually have the ability to “fake” a proof in a way that will require significant effort to detect the fake, are experts themselves, and they usually (though not always) know better and don’t write such papers in the first place.

4
  • 3
    Thanks for dredging up Scott Aaronson’s list. Before anyone complains, these are signs of possible nonsense. Certainly correct mathematics has been written in Word, for example. I have given positive referee reports on papers that did not use standard math notation and did not know how to properly use latex. In one case, the abstract was very clear and paper discussed relevant papers that seems fit for the task at hand. Well worth the effort to get through, and I gave the author pointers on notation familiar to mathematicians. The author was not a mathematician. – Terry Loring Feb 2 at 3:27
  • 3
    @TerryLoring: It's just a list of signs, not a disjunction that implies crackpottery. A good heuristic merely has to be correct most of the time, so that we can combine a few good heuristics to get high confidence of correctness if all the heuristics agree. – user21820 Feb 2 at 3:45
  • 1
    Also, see Spotting Crankery on Math SE. – user21820 Feb 2 at 3:46
  • 4
    In the old days, before the four-color theorem was proved, Frank Harary (the author of a standard graph theory text) received so many alleged proofs of this theorem that he prepared form letters (the then-common purple ditto sheets): "Dear ___, Thank you for your proof of the four-color theorem. The first error is on page __. Sincerely, Frank Harary." – Andreas Blass Feb 3 at 3:24
19

Your question has a somehow detailed answer in the following paper:

W. Hodges, "An Editor Recalls Some Hopeless Papers", The Bulletin of Symbolic Logic, Vol. 4, No. 1, pp. 1-16, 1998.

That paper is dedicated to the many papers the journal received refuting Cantor's proof that real numbers and natural numbers have different cardinalities. It analyses the paralogisms contained in these papers, and I think that what it says can be generalised to other situations as well.

But beware that it's not only maths that receives crackpottish submissions: each field has its targets. For instance, in physics, relativity, quantum mechanics and Newton's third law are quite a target. And the submitters may be professionals and not only amateurs. To give you an example, there is a now retired well-known researcher in my field (metrology) who about twenty years ago developed a "theory" which he thinks should substitute Einstein's general relativity. So, about twenty years ago, he started to submit his work first to physics journals, which outright rejected it, and then to conferences in his own field, hoping for acceptance from his colleagues. This caused some headaches to the organizing committees, which anyway rejected his work (as far as I know, it's not been published so far).

So, as far as I know, most editors probably simply shrug those kind of submissions off.

13
  • 3
    I just read the paper you recommend, very interesting, thank you! However, neither it nor your answer addresses the OP's question: how does a journal react to a crackpot submission. (Unless your answer is that "the editor collects such submissions and distills the lessons learned into a paper".) – Stephan Kolassa Feb 1 at 14:55
  • 1
    @StephanKolassa It depends on what one intends by react. First, at least in my experience, it's the editor who reacts and not the journal, so the reactions are rather personal. The last line covers the reactions of the editors I'm familiar with, at least for my field. There aren't really any more details. The paper instead reports how an editor reacted by analyzing those papers. I'm not aware of different reactions, at least, from the editors I know. – Massimo Ortolano Feb 1 at 16:39
  • 1
    @StephanKolassa: The cited paper here is extremely misleading because the author clearly is unfamiliar with formal deductive systems for basic FOL, in most of which one is permitted to give a name to a witness of an existential statement that holds in the current context (whether via ∃-elim or in the process of deriving the conditions for ∃-elim), which is why he calls a logician Beth "mad" in a cited part of Beth's paper despite it being completely correct!! Anyone can either learn a formal FOL deductive system or check with a professional logician to confirm what I say here. – user21820 Feb 2 at 3:31
  • 2
    In particular, Hodges writes "This last sentence is completely mad. [...] if we know that 'Some individual lives in Neasden', it makes no sense to "agree to give this individual the name 'a' " until we have picked out one such individual. But Beth has done nothing to pick out an individual". This is absolutely nonsense. Removing the ability to assign a name to an existential witness is equivalent to removing ∃-elim in many FOL deductive systems, which is what is truly mad! Beth has no reason or obligation to show how to 'pick out' an existential witness before being allowed to name it! – user21820 Feb 2 at 3:39
  • 1
    @user21820 I read the paper, and agree with you that that part doesn't seem to be correct. But the rest seems okay. So I'm not sure whether Hodges was showing (unintentionally) the very same fallacy that Hodges is refuting in that section. – justhalf Feb 2 at 9:35
7

I'm not a mathematician and I never saw this happen myself, but I heard from a friend that the policy of their department was that if a crackpot paper was received they gave it to a grad student to look over. The instructions to the grad student were "Read it until you find the first error, then STOP, write a quick summary of the error, and then send it back". I think the grad students took turns with this.

I don't remember why but apparently their department tended to be a magnet for papers about a particular topic (possibly the Riemann Hypothesis thing, maybe? I don't remember and am not a mathematician) so any paper coming in on that topic got the above treatment.

4
  • Let's hope the grad students did get properly paid for that! – user111388 Feb 2 at 7:16
  • 1
    @user111388 They surely did - they got a lot of training finding errors :D – Zizy Archer Feb 2 at 9:13
  • 1
    I've heard this attributed to Eric Temple Bell (in Ian Stewart: "The Problems Of Mathematics"), with reference to Fermat's Theorem. – Ulrich Schwarz Feb 2 at 16:44
  • 3
    Precisely this was (part of) my job as a TA. My thesis advisors name was quite established in number theory, attracting a steady stream of papers on the Riemann hypothesis and Collatz conjecture, among others. I got more than fair compensation, but at times I would question my sanity; finding the first error in a text that has no discernible premise, argument or conclusion can be maddening! – Servaes Feb 3 at 21:00
-5

There is always the possibility that some gem is hidden in what otherwise seems like a crackpot submissions. An editor has to accept the that a field can be revolutionized by a single new idea.

A good editor knows most of the ways that thinking goes astray and can categorize wrong ideas easily and tell the author where their own thinking is wrong and respond with stock responses they've already collected. When they can't do this, either the individual is onto something new or a new path of wrong thinking has been found and the editor can update their set of premade responses. In either of these cases, the world gains.

If you can't do this, you've succumbed to the hubris that established power often falls into and it serves no one, ultimately. Honestly, unless it's vandalism or disrespect of the establishment, people should be worthy of responding to.

If we have a society where people are too worthless to give criticism or feedback, then a Doctor of Philosophy has to step up and understand what's going on in their society.

3
  • 2
    Sounds like you haven't dealt with crackpot papers in real life. – Martin Argerami Feb 3 at 12:20
  • 4
    This is incorrect. As a grad student I was asked to do peer review for around a dozen crackpot papers, including proofs of RH etc. It's not always easy to pinpoint an error; rather, the reasoning will often be muddled and imprecise, and it won't necessarily be clear what the author is thinking or if they actually understand what they're claiming. – academic Feb 3 at 13:07
  • @MartinArgerami: I deal with the same problem everyday: the internet. – TheDoctor Feb 4 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.