70

Note: This question is intended to be a generic version of something that gets asked occasionally on this site. I apologize for making it overly-contrived, but it's intended to be a generic template of this style of question. A previous version of this question was locked due to Personally Identifiable Information (PII), so I'm re-posting the question anew.

I was recently reading about [topic] and I kept finding articles by [crackpot], such as [link] [link], and [link]. How the heck can anyone trust this person? They've been disproven as a crank by [notable person] and [notable news source], as well as people on our own website: [link to some other stackexchange site]. People tend to say really nasty things about them:

[crank] is an idiot who only spouts [expletive]. I wouldn't ride the bus with them, much less be a collaborator on academic work.

How on Earth are trusted academic sources such as [high h-index journal] publishing work from this [amusing yet insulting word]?

2
  • Comments are not for extended discussion; this conversation has been moved to chat. Please read this FAQ before posting another comment.
    – Massimo Ortolano
    Jul 7 at 19:28
  • 37
    I'll just say that I think about peer review the way Churchill thought about democracy: It's the worst possible form of scientific review, except for all the rest that anyone has ever come up with. Jul 7 at 20:37

13 Answers 13

88

I'll note that a "crank" in one field might well be an expert in another. Some incredible racists who spout crap are well respected researchers in, say, math or electronics.

I'll also note that someone who has done important work early on can become a crank later in life. The opposite is also true. Road to Damascus, and all that.

Sometimes a perfectly well respected and published researcher will run in to some idée fixe that they fail to shake off with all of their attempts to follow through rejected by their peers.

I'll also note that the incompetence of reviewers at good journals is probably the least likely explanation. And pressure on reviewers, likewise, is hard to manage with blind reviewing.

But, was Poincaré a crank for not recognizing Einstein's work? Or Einstein for turning aether theory on its head?


Note that neither the question nor this answer address the issue of how crackpot articles get published. That requires a completely different analysis.

2
82

Why would it be impossible? Remember, peer review is only supposed to judge the work that is presented. It's not supposed to judge the reputation of the authors, or the credibility of the author's work that wasn't submitted.

Take any of the papers you linked, then cover up the names of the authors and show it to people in those fields. Will they disapprove of the work? If they only start disapproving when they know who the author is, then we have exactly something which double blind peer review is supposed to stop.

The idea that [crank] is a crackpot and therefore everything they've ever written is untrue falls afoul of ad hominen. Beware.

10
  • 1
    On the other hand, the "stopped clock" principle applies. Papers are normally not just expected to show results, but to show that those results prove/disprove/give probability bands for some hypothesis. Accepting a paper doesn't just mean endorsing the results, but also endorsing (at least to some extent) the hypothesis.
    – Graham
    Jul 7 at 12:44
  • 18
    ... It's also important to note that if someone has a track record of "research" which is structurally flawed or simply inaccurate, then we do genuinely need to distrust everything they do. There have been various high-profile researchers who have been found to have falsified results, resulting in all their work for years/decades needing to be rechecked. "Ad hominem" only applies if the person is working competently in good faith and has their work rejected for non-technical reasons.
    – Graham
    Jul 7 at 12:47
  • 4
    @Graham Falsification is a very different problem. Structurally flawed research is readily detected by peer review.
    – jakebeal
    Jul 7 at 15:12
  • 5
    @jakebeal It's a different problem, but either way it comes down to whether the person is engaging with the scientific community in good faith. "Crackpot" ideas aren't those which go against current opinion, but more those which are defined by ignorance or misrepresentation of the field in general.
    – Graham
    Jul 7 at 17:27
  • 2
    @Graham indeed, and the solution to this problem is not "reputation-based gatekeeping", but requiring submission of all data/code used so that it can be scrutinied, and expecting reviewers to actually have a look at it. (which obviously is easier for software-based experiments, less so for wetware-based stuff ... but replicability works quite differently in such projects in the first place, and presumably similar QA checks can apply there as well). Jul 8 at 11:52
18

First, I think it is appropriate to describe peer review by paraphrasing Churchill's famous quote on democracy: It is the worst form of publishing scientific results, except for all the alternatives. Peer review has a high failure rate. I've seen results that can be easily demonstrated to be fundamentally flawed, containing elementary errors in math or reasoning, appear in top-rated journals. I fought and sometimes lost battles with referees whose criticism was obviously nonsensical, but they stuck to it, and eventually I just took the coward's way out and submitted the paper to another journal. Still, when you look at what gets published when there is no effective peer review, clearly peer review helps a great deal, its flaws notwithstanding.

Second, there is a continuous spectrum between mainstream science and utter crackpottery. Revolutionary ideas sometimes appear at first in a form that makes you wonder what the author was smoking. Consider this quote from James Joyce, for instance, "Three quarks for Muster Mark! / Sure he has not got much of a bark / And sure any he has it's all beside the mark." It inspired Murray Gell-Mann to use the word "quark" in his proposed mechanism behind the "eightfold way" involving a new substructure for baryons and mesons, with elementary particles carrying fractional charges. A miracle this paper even got published. Yet it now forms the foundations of the SU(3) part of the Standard Model of particle physics, one of the crowning achievements of modern physics overall.

Finally, it is important to note that no crank or crackpot thinks he is a crank or a crackpot. I am regularly approached by strangers offering their, ahem, unconventional ideas on physics. Many have respectable scientific credentials. They have strong faith in the validity of their concepts and often go to great lengths to explain them in detail, "prove" them, provide background, even propose experimental verification. Most of them are genuinely good, well-meaning people, devoted to what they do, who spent a huge amount of time developing their ideas. This, of course, makes it even harder for them to accept the possibility that they were wrong all along, that what they saw as profound insight was just a symptom of their profound ignorance.

Which sometimes makes me wonder: How do I know that I am not one of them? Indeed, how do you know that you are a crackpot? The answer is, you don't. It's like that German Autobahn joke I heard eons ago, about a driver who listens to his car radio warning travelers that a demented driver is going against traffic on the wrong side of the expressway. "One driver?" he asks rhetorically, pointing through his windshield, "There are hundreds of them!"

5
  • -1 due to the Churchill quote, which, if applicable, would only cast doubt on the peer review system.
    – einpoklum
    Jul 9 at 20:45
  • 8
    The intent was precisely the cast doubt on the peer review system, as it is far from perfect. That is not to say that I know of a better alternative... which is exactly what the Churchill quote was intended to express. Oh well. Jul 9 at 21:59
  • 1
    Well, Churchill's quote is a generic claim that anybody would make about the system they support. A Monarchist would say Monarchy is the worst form of government except for all of the alternatives.
    – einpoklum
    Jul 10 at 8:42
  • 3
    I respectfully disagree. I would only use this quote to describe systems that I consider flawed, perhaps even deeply flawed, yet with no better alternative available. It is an acknowledgment of the flaws of the system one supports. Jul 10 at 13:51
  • 1
    +1 for the Churchill quote 🙂
    – arni
    Jul 23 at 12:04
16

Just as an added note: if not all research of said crackpot is bad (and as others have said, it most often isn't), if the review process is double blind, then the only person who could reject the paper simply because they knew the person was a crackpot is the initial editor who accepted the paper for the peer review process. And I really don't know if editors make it a habit to look up every single author of received papers (I guess they rather don't).

2
  • 5
    And there is an important question - even if the editor "knew the person [is] a crackpot", should they do anything special at all? If a quality, double-blind process feels the article is worthy of publication, why over-ride it? Lost in the genericized question is whether the contents of the article is crack-potty, or merely, elsewhere, one of the authors. Excluding adjudged quality articles based on authors' other publications is but a short distance to "no, we won't publish your article in our high impact journal because you're not prestigious enough!"
    – Houska
    Jul 7 at 19:01
  • 1
    The editor should look up the author's previous work, even if only briefly, to see if the work is "salami slicing" or has been published already in a too-similar form. Not to do so would be wasting the reviewer's valuable time and energy. Jul 23 at 6:36
10

Jacques Benaventiste published in Nature an article about (broadly speaking) the "memory of water" in Nature. Nature added a warning about "the incredulity of the many referees", but still went for the publication.

Why? Because Benaventiste was a Famous Scientist That Can Be Trusted (TM).

For anyone else, the article would have been rejected with a roar of laughter, but here rationality went on vacation and Nature wanted to be the first one to publish breakthroughs.

So to answer your question: this happens sometimes because of money and connections.

A special mention to Luc Montagnier, whom I admired and was proud of his Nobel Prize, until he went nuts about homeopathy and vaccines (including his defense of Benaventiste's biased results)

11
  • 1
    This seems to be answering the question "how does a crackpot paper get published" whereas the question was more about a person. In reference to the original question asked, the answer supported by what you write here isn't "because of money and connections" but rather "because the previous work by the crackpot was legitimate work, and the crackpottery came later".
    – Bryan Krause
    Jul 7 at 16:36
  • 1
    @BryanKrause well, he was published because he was well known (admittedly, though his previous publications, but also his professional activity). This means that Nature trusted that past more than their own referees. I call that "connections" but it can be also called "blind trust for someone well known". As for the money - I was referring to the money Nature would make (in the future, as a top journal) by publishing breakthrough results despite the universal critique. They could have rejected it, on the basis that extraordinary claims require etc.
    – WoJ
    Jul 7 at 16:42
  • 3
    Nature writeup on this story. The assertion that this is based on "money and connections", while possible, is highly unlikely; Benveniste was a highly accomplished immunologist, well-respected in his field when this was published. The research was eventually disproven, which is actually evidence that the system works, not the other way around.
    – eykanal
    Jul 7 at 21:22
  • 3
    @eykanal: Nature published the paper because they would be THE ones who published a breakthrough if it was real. The fact that the claims were extraordinary with little to no physical backing did not stop them from doing that. Their referees were against it. Would my paper about that be publshed? No. Why was his paper published? Because he was well known. In other words, no matter what he wrote, even if it made little sense, he had the privilege to be published outside of standards. I call this "connections".
    – WoJ
    Jul 8 at 8:34
  • 1
    @Taladris - Read the abstract; it's a paper on vaccine dose concentrations. "Memory of water" refers to the fact that he claims to see immune response even when the vaccine is diluted so much there's not even a molecule present in the actual administered dose. He claims the solution had "memory" of the vaccine; hence, "memory of water".
    – eykanal
    Jul 9 at 21:01
7

The original posting had links to articles with multiple authors. Where there are multiple authors, it is possible that the articles are fine and that most of the authors are fine, but a person somehow insinuated his way into being listed as a co-author but in fact did little or no work on the article. This could happen in a variety of ways from just being friends with the authors, being persistent or doing some favor. I assume that journals accept the list of co-authors submitted, and do not investigate each to see if they actually made -- or could have made -- any substantive contribution.

6

There is of course the alternative possibility that the person being pilloried by

[notable person] and [notable news source], as well as people on our own website:

may simply be correct. It's not a priori reasonable that those authority sources deserve more faith than the published researcher on the topic in question.

People tend to say really nasty things about them

And this is sadly often true about those going against the majority view, the existing power structures, groupthink or vested interests (classic examples: Giodarni Bruno, Galilleo, Darwin).

One can receive opprobrium and still be be correct. If the anonymous work has been peer reviewed and stands up on its own merits, then that may in fact be evidence that the criticism received by the

[amusing yet insulting word]

may not be entirely justified. Suppression of dissenting ideas has often slowed new discoveries and scientific progress, so we should be wary of contributing to it in our own modern way.

6

They've been disproven as a crank by [notable person] and [notable news source].

Because some basic logical fallacies are assumed in such a subjective question, most any answer would have to first address that. They are:

  • Argument from authority
  • Fallacy of incomplete evidence
  • Ad hominem

Addressing evidence for or against the specific claim rather than spending time on reputation of the claimant is the only way to determine the claim's veracity.

5

One possibility is that the crackpot is a close friend of an editor in chief of a peer-reviewed journal. The editor in chief unethically changes the peer-review process so that the paper is accepted even if it is not worthy of publication.

6
  • 7
    Possible, but incredibly speculative.
    – cheersmate
    Jul 7 at 7:40
  • 3
    @cheersmate My answer is based on a real situation but I don't think it is appropriate to give details here. Jul 7 at 8:27
  • This seems like a road to the end of the line for that journal.
    – Buffy
    Jul 7 at 14:50
  • 1
    @Buffy: Not true. It's precisely because people who know about such journals are not revealing the details that they won't die out so easily.
    – user21820
    Jul 7 at 17:49
  • 1
    I can also give an examples of "crackpot" papers by members of the editorial board of the journal in question. It isn't speculative. Jul 22 at 8:35
3

One of the reasons why "crackpots" can get papers published is because a lot of journals allow authors to suggest potential reviewers. Journals should never do this - the editors should be sufficiently familiar with the topic to be able to select appropriate reviewers for themselves (authors should be able to suggest people that shouldn't be selected as reviewers). I feel extremely uncomfortable every time I am asked to suggest reviewers for my papers, but happily this is fairly uncommon in my field (machine learning).

I (or at least my fictional alter ego, Gavin Cawley) was involved in a case where this led to a comment paper by the journal editors explaining the failure of peer review and the change in the journal's policy (it wasn't the first time this had happened).

Another reason is the lack for academic reward for publishing comments papers, which should be an important element of post-publication peer review or quality control. If people knew their "crackpot" paper would be likely to attract a critical comment, there would be more of an incentive not to do it. I've written a few comments papers and they are a lot of work.

Journals don't seem to do a great deal of checking to see if arguments have been made and refuted before. One of the comments papers was on a study that argued a statistic technique used in a variety of areas in biology was wrong. But of course, it wasn't, the authors just didn't understand it correctly. So I wrote a comment paper, but I have found the authors have published a large number of papers making similar claims in a variety of applications of this technique. Nobody has the energy to refute all of them.

The last reason is that there are a lot of journals these days, so if you get a paper rejected, they can easily be sent somewhere else, and if you test the lottery of peer-review often enough, it will eventually fail. Good authors adapt their papers according to the reviewers suggestions, bad authors just send it off somewhere else with minimal changes.

So there are a couple of practical reasons why it happens.

Some have mentioned Galileo and Darwin and Einstein. It is important to consider though that Galileos, Darwins and Einsteins are vanishingly rare, but crackpots are near ubiquitous. So if you think you are a Galileo the odds really are not in your favour and it is self-skepticism that stand between you and "going emeritus".

1

Among other things, based on observation, sometimes an element of the crank-ery is an exorbitant self-confidence and enthusiasm for self-promotion... even, or perhaps especially, in the face of negative professional feedback.

I can easily visualize personally timid expert people getting worn down by the importunings of a crank they misguidedly tried to help.

I have a little more difficulty visualizing serious journals getting similarly worn down, but it's conceivable to me that it could happen through various event-sequences. "Being pushy" does have its rewards...?!?

I think the relevant dynamic here is that "the crank" "has nothing to lose", and subliminally realizes this, and thus has a much different context in which they operate. The established professionals and journals do "have something to lose", but apart from inhibiting endorsing/publishing doubtful things, there is also a strong social/moral disincentive to be toooooo negative to enthusiastic amateurs, etc. This dynamic is used by con artists more generally, I gather. Playing on the tendency of people to "be nice".

4
  • 1
    I will not deny that I do not like how the question is phrased and I like this answer even less. There is an entire spectrum of "cranks", going all the way from insanity through carelessness to something else entirely: en.wikipedia.org/wiki/Abc_conjecture. The answer to the closed question was substantially better. Also, in my humble opinion, in non-axiomatic sciences, the distinction between a crank and an established researcher can often come down to a matter of opinion/education and personal relationships/like/dislike. Jul 6 at 23:15
  • 3
    I'm curious whether this is something folks have experienced. I haven't seen this sort of thing when reviewing... personally, I'm comfortable giving harsh reviews when necessary, and I really haven't seen editors getting "worn down" and letting inappropriate things through.
    – eykanal
    Jul 7 at 2:33
  • I've never seen something really bad getting through by wearing down the editors/reviewers. I've definitely seen that happen with things that were basically worthless due to their mediocrity. Not wrong, just pointless...
    – jakebeal
    Jul 7 at 15:15
  • Being successful in science requires both self-confidence and self-skepticism, you really do need both. Self confidence is fine, providing it doesn't stop you from taking criticism seriously. Jul 22 at 8:40
-1

There was this one guy called Galileo Galilei who was considered a crackpot, and a dangerous one, by all the experts at the time he was alive.

He was later proven to be correct, but there are still people believing the erstwhile experts to this day.

He eventually got published, despite being considered a crackpot...

3
  • 3
    There were these hundred of guys who considered themselves a Galileo Galilei, but all of the experts considered them crackpots, if they could be bothered to consider them at all. They were never proven correct, but still believe they are a Galileo Galilei to this day. They never were published, despite their belief that they should be. // For every unrecognized genius, there are a thousand crackpots.
    – henning
    Jul 21 at 12:14
  • 2
    As Carl Sagan said "But the fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown.” The experts are right the vast majority of the time, the rate at which scientific paradigms are overturned is positively glacial, and with good reason. Jul 22 at 8:38
  • 3
    BTW I don't think Galileo was considered a "crackpot", the problem was heresy, not crackpottery. Jul 22 at 8:42
-3

Others have made valid points here, but there is one that seems to be missing. Or, at the least, talked around. And that is the logical fallacy you are engaging in here.

One should never, ever, ever judge a factual, or logical argument based on the person making it. Merit alone is all that matters in the world of knowledge. That is known as the Ad Hominem logical fallacy. All other reasons above are valid. But to those I'd like to add that even an idiot can be correct. Hell, you've heard the saying that a broken watch is correct twice a day.

To put it another way, if a mentally handicapped person (once regarded retarded) were to tell you the sky was blue, would suddenly doubt a lifetime's experience? If a known idiot told you that 2 + 2 equaled four would you assume it equaled 6?

No, of course you wouldn't. Because all that matters are the facts.

And this goes the other way as well. Just as it is a logical fallacy to assume anything a crackpot says is wrong, so too is it a fallacy to assume anything a respected researcher says is correct.

We see this in politics all the time. And it goes beyond simply ignoring anything a Democrat or Republican says simply because they are a Democrat or Republican. It goes beyond ignoring Fox news or MSNBC. Both sides love to hurl names at each other, negatively charged descriptors, very much along the lines of the one you employed: crackpot. But to that they add Nazi, racist, rapist, heartless, supporter of big business etc. etc. etc. All that should matter are the details, ironically the very thing many politicians would die before giving you.

Nor can we assume that, just because someone has a high degree they will be correct. The tobacco companies proved that one conclusively. Sadly, I see this form of the Ad Hominem fallacy the most. The entire world seems to suffer from Wizard of Oz syndrome, acting as if it is the attaining of a piece of paper that somehow imparts knowledge. In truth, the paper is meaningless to any discussion. It is the points presented, and only the points presented that should matter here.

We want the facts, and nothing but the facts, so help us god.

7
  • 2
    I strongly disagree with this answer. Suppose I've never encountered Special Relativity. Then my crazy Flat-Earther uncle tells me that time slows down on objects travelling close to the speed of light. It doesn't make sense to take him seriously. But if my university physics professor tells me the same thing, then it's worth looking into. Jul 9 at 4:53
  • Well, you too have committed an ad hominem logical fallacy. What you should have done when your crazy flat earther uncle told you about relativity was to try to corroborate his data, not simply assume he was wrong. Would you have dismissed what he said because he looked at his watch and told you it was 5:32? Would you have assumed that anything he ever said about anything was wrong because he was a flat earther? Jul 10 at 21:11
  • 3
    No. People study hard and gain credentials in an area, and are eventually seen as "experts". So I'll trust my physics professor more than I'll trust my crazy uncle. Because nobody has the time or the resources to check absolutely everything that anybody says to them, in a vain attempt to avoid ad hominem fallacies. Are you seriously suggesting that if my uncle tells me that Australia doesn't exist, and all Australians are paid actors, that I should buy plane ticket and go there, just to make sure? Jul 11 at 20:26
  • 1
    "One should never, ever, ever judge a factual, or logical argument based on the person making it." no, this is obviously not true, simply because we have to make judgements to a time and energy budget. Whether an argument is correct or not does not depend on its source, whether we should invest time in the argument does. Jul 22 at 8:10
  • 1
    How do you chose your dentist? Do you trust their expertise and education, or do you just give a random person a try? Or conversely, do you have your dentist explain every bit of the procedure and go home to check in detail whether what they propose is the right thing to do, or do you trust them to do what's right according to good medical practice (which doesn't exclude consulting you for your preferences etc. when that's relevant)?
    – henning
    Jul 22 at 12:32

Not the answer you're looking for? Browse other questions tagged or ask your own question.