What is the difference between falsifying data and just being wrong?

Question

I had to take a standard ethics seminar at my research lab (everyone in my lab must take the seminar). As one would expect, I learned that the charges associated with falsifying data are serious.

Whether you falsify data or make a stupid mistake, the end result is that the published work is wrong. But the charges of falsifying data are much more serious than the charges associated with human error.

My motivation for making this post is as follows. Researchers must be allowed some leniency on accuracy, otherwise one would not progress at a reasonable pace. Nobody is right 100% of the time. Some of the best papers and discoveries have inaccuracies (some are in fact 100% wrong). However, if your data is innacurate, and the allegation is that you falsified the data, your career takes a major hit.

Question

What are the specific differences between falsifying results and making a human error? How does one decipher between erroneous results (due to human error, or negligence, or incompetence, or an early career mistake, etc) and falsified/fabricated results? How does one prove results are falsified? Conversely, how would a researcher prove his/her innocence?

Answer 1

The clear difference is intent.

Human error occurs when people make mistakes accidentally. For instance, in my first paper, I submitted a graph that was physically implausible because of an error in a code I had written, without realizing it at the time.

However, if I knew the work was wrong or, worse still, if I intentionally made up results, we would cross the boundary from human error to fraud.

Unfortunately, it can be very difficult to detect intentional fraud without some assistance from the perpetrators themselves. A major example of this was the Jan-Hendrik Schön scandal of the early 2000's. Schön's fraud was discovered because he was reusing the same figures to different ends in papers in various journals.

Answer 2

As @aeismail says, the difference between falsifying data and just being wrong is intent.

As for how one can judge this, often you simply can't, and your own question provides a rather nice example. Daniel Collins in a comment mentioned that he suspected your question was posted out of a desire to learn how to game the system so that you can apply what you learned to actually game the system. On the other hand, it seems totally possible that the question is legitimate and motivated by a genuine desire to understand how science works and how fraud in academia is detected and handled. So you see, we cannot judge and cannot accuse you of having bad intent with a high level of confidence, but can only raise suspicions.

Similarly, when wrong results in science can be plausibly attributed to error, that is usually the interpretation that will be assumed. People usually don't go around accusing others of fraudulent intent to falsify data unless they have some pretty convincing and damning evidence. Smoking guns are rare, which is why you don't hear about such cases very often. On the other hand, we do occasionally have such cases, since people who falsify data are usually not very smart or sophisticated to begin with, so often they will be sloppy and careless and go about their data falsification in some very obvious and easy-to-detect manner, and then they are discovered.

Answer 3

You should document what you have done while you are doing it, e.g. in a research log. You should make that documentation available at the same moment that you publish your results. If a wrong result happens, then it will usually be possible to track down in the documentation where the mistake happened. That will make it more plausible that it is an honest mistake.

Answer 4

Goldfinger's law:

• Once is happenstance
• Twice is bad luck
• Thrice is enemy action

Intention is something that can be proven over a lingerer period; an isolated mistake in a development is usually not interpreted as intentional. Nevertheless, even isolated honest mistakes can cost you your reputation, especially in high-stakes research results. Fleischmann & Pons in "cold fusion" is an example; I do not know what happened to the reputation of the Gran Sasso "Faster than Light Neutrinos" group, but I am sure their mistake didn't help.

Sometimes, there are tricks to make intentional scientific fraudsters reveal themselves, but I won't mention them here to not render them ineffective. But a famous example you can read up about are the N rays. Also, some nice chapters in the (first) Freakonomics book about made up results and rigged fights.

For anyone contemplating faking results: what'd be the point of it? Why not going to politics, stock markets, poker competitions, army - that's where generating misleading information is desirable and part of the job description. But in science, we are in the business of truth-finding.

Answer 5

There are circumstances where incorrect results are published and it is clearly and unmistakably the result of academic falsification (although I would concede that perhaps only a small percentage of all academic falsification ever gets caught). Of the instances of academic falsification that gets caught, they fall into a few (but growing) catagories:

1. Image manipulation. There is no logical or scientific reason for copy/pasting a band in a gel other than to mislead. This is the closest thing we have to a smoking gun, because the data is obvious without any kind of statistical analysis needed. Here is an excerpt from a recent anonymous peer review via pubpeer:

2. Unusual statistical practices. This is particularly true in RNA-Seq experiments, when there are multiple ways to skin a cat. However, every now and again someone conducts their analysis in such a convoluted way that one can only assume it was an attempt to deliberately mislead. I'm not talking about applying the wrong statistical test, i'm talking about using different statistical tests in the same paper for the same type of data. I'm talking about arbitrary cut-offs to 3 decimal places which just-so-happen to produce a statistically significant result, but any other value for that cutoff would have produced a non-significant result (p-hacking). Essentially here we imply motive due to incredibly improbable actions taken by the authors in how they reasoned about their data.

3. Whistleblowers. Occasionally, and more frequently, we see collaborators or individuals who left academia for one reason or another coming out and proclaiming that they know such-and-such a publication was fraudulent because they actively participated in it. Unfortunately, this scenario is not very effective at getting a fraudulent publication retracted, as many publishers and academic integrity investigators can brush off such reports as hear-say or that the whistleblower has some ulterior motivation. This method, however, does have the ability to expose types of fraud that are impossible to detect in a publication - such as samples excluded from the analysis that would have otherwise contradicted the published result.