13

Many papers use nonstandard data sets, which means the authors could have manipulated the data behind the scenes to make their paper look more impressive. It seems that this could be done very easily: for example, one could cherry pick the items of the data set to make it seem like an algorithm has 90%+ accuracy while in reality it's about 60%.

What prevents people from doing this? Is there something I'm missing?

  • 14
    Sure, you can, and if done correctly it will work at first. But when you are caught, your reputation as a scientist is lost forever. As a wise man said, you can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time. – Federico Poloni Apr 5 '17 at 20:25
  • 11
    The content of this question seems reasonable to me (a little naive, but a sensible question), so I assume the downvoters are objecting to the phrasing. I'll try editing it; feel free to revert if my edits don't capture what you intended. – Anonymous Mathematician Apr 6 '17 at 0:52
  • 2
    If you're willing to lie and have built up a reputation as a scientist over decades then you can get your papers published in the most prestigious journals. The problem is, that you need to a) Have a reputation to lose in the first place and b) Risk losing it when people can't replicate your experiments. – Valorum Apr 6 '17 at 8:37
  • 2
    @LeonMeier I wouldn't necessarily disagree, but most "non-reproducible junk" is probably the product of straightforward mistakes rather than fraud. As somebody said in a different context, "never attribute to malice that which can be explained by stupidity". – alephzero Apr 6 '17 at 9:00
  • 2
    I don't think this is a duplicate to the linked question. The answers here show that the deterrent is not the detection method - discussed and asked for in the linked question - but rather the consequences to career and the lack of a longtime benefit from manipulation. – Grebu Apr 6 '17 at 11:42
36

Can I use what I described above to publish a paper without much hassle?

Yes, it's easy to write a fraudulent paper, and it's not hard to get it published if you are willing to publish in a journal with low standards. (Some predatory publishers will publish literally anything if you are willing to pay them.) You might even get it published somewhere respectable if you fake everything convincingly, by getting results that are good enough to be impressive without being so implausible that reviewers become suspicious.

Is there something I am missing?

I'd bet that what you're missing is the role of reputation in academia. If you don't maintain a good reputation, you'll be ignored and marginalized, with no ability to influence the field. If your only goal is to produce a paper, then there's a lot of scope for mischief, but this generally won't lead to a successful research career. If people are unable to build on your work, they'll become suspicious and your reputation will suffer. (And if they aren't even interested in trying to build on your work, then by definition you aren't having an influential career.)

It's not impossible to succeed in academia by fraud, as demonstrated by the people who succeeded for some years before they were caught. However, it's much harder than faking a single paper.

| improve this answer | |
  • 4
    Very nice answer. One could maybe add that more and more journals require/encourage side by side publication of primary data. On the other hand, one needs to be aware that a certain part of scientific publications is faked. I think I remember a systematic too optimistic reporting on new drugs. They are almost always less effective in the long run than reported initially when they are discovered which seems to indicate maybe not outright data manipulation but at least biased interpretation of data. – Trilarion Apr 6 '17 at 6:28
  • 1
    @Trilarion Medical research is just crazy in general. A lot of these weird claims that x can lead to y are based on flimsy data, but it makes great headlines so those are the "discoveries" we hear. It's kinda scary. – JMac Apr 6 '17 at 10:41
14

What prevents people from doing this? Is there something I'm missing?

In addition to the factors mentioned in Anonymous Mathematician's excellent answer, I think another factor at play is that while it is not very difficult for a dishonest person to (temporarily) get ahead by faking data, the rewards for doing it aren't very great compared to the effort it takes to implement this strategy successfully. Even the barrier to entry in academia, getting a PhD, is high enough to deter the most dishonest people, who are the kinds of people who may be tempted to use fraudulent strategies to attain professional success, money etc. And the payoff for this kind of fraud is ultimately quite modest -- you might get a job with tenure, but is that really worth the trouble of working on something you have no real passion for, and doing so in a dishonest way, for so many years?

The bottom line is that Academia is simply not an attractive place for fraudsters, certainly much less attractive than other industries or career tracks (which I won't name) where fraud and other antisocial behavior can reap you much greater rewards in a much shorter period of time. So yes, a hypothetical fraudster who ends up in academia can gain some traction in their career using dishonest methods (until they are caught at least), and occasionally we hear stories about such people. But these stories are rare precisely because academic research as a career option is most appealing to people who by and large are passionate about advancing human knowledge, and therefore are not predisposed to committing major acts of scientific fraud.

| improve this answer | |
7

An example is Jan Hendrik Schön. He manipulated data on a big scale. But after people found out about it, he lost his job (and academic career) and even got stripped of his doctor title.

Apart from this, I guess that most scientists are actually interested in honest academic research (i.e. finding out something truly new) and not in a career build on lies. Moreover, many scientists suffer from some other extreme, they believe that they are imposters (while producing valid science).

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.