6

Are there any research/study/survey that tried to analyze whether industry-sponsored papers contain more incorrect results than papers that aren't industry-sponsored, and if so, to what extent?

For example, if a drug company publishes a paper about the efficiency of one of their drugs, some readers might suspect results are more likely to be positively biased than if the paper was published by an academic lab with no funding from the drug company. I wonder if these kinds of suspicion are actually grounded.

  • 1
    While the fact seems likely, this study would heavily rely on subjective and anecdotal evidences, and no new conclusion could be drawn from it. A thoroughly unscientific study. (And I know the world is full of studies consisting of such useless statistical surveys.) If you are suspicious about any paper, investigate the specific case. – Karl Apr 22 '19 at 10:36
  • 1
    Define an industry-sponsored paper? Do you suspect that the huge corpus of work from AT&T Bell Labs is more flawed than from some university? How do you determine that with enough statistics to really tell? (I would note that examples such as you gave are the reason for conflict of interest statements, so agree that it has happened in some fields.) – Jon Custer Apr 22 '19 at 13:18
  • 1
    This reminds me of the Bayer and Amgen (companies) studies where they claim that they could not replicate approximately 90% of the published biomed papers they tested - but they did not publish what papers they attempted. Reference that notes this: sciencemag.org/news/2017/01/… I think the real problem would be identifying what papers are actually industry sponsored, as often this is unknown and only would mean directly sponsored - indirectly sponsored projects are hard to identify. – BrianH Apr 22 '19 at 16:04
2

I think your hypothesis is implausible that industry-sponsored research is more incorrect or less reproducible than non-industry-sponsored. Or can you give some reasons for this assumption?

On the contrary looking at many of the meta-surveys (partly conducted by industry, because non-industry-sponsored research maybe is less reproducible to their experience) that investigated the reproducibility of academic research in the recent years, rather the opposite is maybe the case and known within industry.

Additionally, I don't see how it makes a difference if money financing a PhD student comes from industry or not? Wouldn't the rationale rather be, if money is spent by industry, then rather PhD scientists/Professors are hired for a project than cheap and still learning PhD students pressured by publish or perish? And to my knowledge more of the spotted scientific misconduct has been caused by untenured researchers. Again, publish or perish

And when industry research can't reproduce 70–90% of academic laboratory findings, then a better question is probably if most of the research projects are funded by industry or not and what kind of researchers are on average hired for industry-sponsored projects (PhD students, PhD's, tenured)

What maybe sounds plausible, that researchers carrying out industry-sponsored projects tend even more to make up results or buzzy headlines ("new battery technology allows charging up within 10% of normal time" which I read every month or so), is rather implausible to me, because important and valuable results in industry-sponsored research projects are often rather published as a patent than a paper.

More importantly, there is a clear correlation that research in branches being strongly entangled with industry (engineering, physics,...) is much better reproducible than weakly linked branches (social sciences, psychology, biomedical,...). Most of the meta-surveys are in such fields.

To take up your case of drug development, such only become legalized after 10 years of clinical trial, so I think the incentive for some PhD student is not significantly higher and often the PhD students have no clue if the money financing them comes from industry or public. And professors being biased by the funding source mostly don't conduct and evaluate the experiments and would have to alter the results (which also happened, but rarely)

Summa summarum I think there is no such study (my googling didn't find one but many meta-survey on reproducibility partly published by industry companies), as your hypothesis looks very implausible to me. There is probably a good virtual database with so much meta-surveys and the necesssity to name funding sources in a paper, still to me the important question would be, which kind of researcher (students, tenured,...) are typically hired for non-/industry sponsored projects? Because, the conclusion from the published meta-surveys in social sciences, psychology, biomedicine and the low reproducibility to be drawn is not that researchers in such fields tend to be more biased, tricking, cheating and therefore more incorrect results are published, but rather that industry is not much interested in such results and the level of scientific rigor is much lower than in hard sciences like quantitative physics or engineering, for which I didn't find a single meta-survey.

| improve this answer | |
  • 1
    I don't know that the OP was specifically pushing the assumption. It is certainly something we see a lot nowadays with climate research for example. Data would be interesting. Though good point about the difference of fields. Industry probably is far less likely to waste their time and money with nonsense such as low power studies to analyze whether rich people are bigger jerks. So it would be hard to make a fair comparison of researchers. – A Simple Algorithm Apr 22 '19 at 22:47
0

What is a "result"?

Many industry papers I've seen do not contain any hard results anyway, but just report some andectodal evidence how they improved some software they used. I doubt there can be anything "wrong" there.

It's not as if they contained plenty of wrong proofs.

| improve this answer | |
  • I think the question gives one example of results. If some papers don't have any results, then we can ignore them. – Franck Dernoncourt Apr 23 '19 at 7:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.