-6

After many years of doing medical research it is very clear to me that the vast, vast majority of what we create in terms of research is utter garbage, and it is so a priori, i.e. we know it before we publish.

There are hundreds of meta research papers on this topic, including the Lancet series “Reducing Waste”.

Why is this the case and how can this be resolved?

19
  • 5
    Surely this question must be answered by the Lancet series you cite?
    – avid
    Sep 26, 2023 at 7:03
  • 10
    I think you would do well to explain why you think it’s ‘utter garbage’ and where you arrived at the ‘90% useless’ figure
    – user438383
    Sep 26, 2023 at 7:59
  • 5
    Research has shown 90% of research is useless. Though there are some caveats: 1. We do not know which 90%. 2. We do not know if above research is part of the 90%. 3. We do not know what "useless" means, anyway. ("What's the use of this thing called electricity?" "Number theory being useless" [Hardy], etc.) Sep 26, 2023 at 10:34
  • 4
    "we know it before we publish" - the use of "we" suggests that you see yourself as part of it, so some introspection might help to answer your question. Why do you do it? Why do your colleagues do it? What would stop you or your colleagues from doing it? Sep 26, 2023 at 10:38
  • 4
    It is an egregious misreading of Ionnidis to infer that 85% of research findings are known to be "utter garbage" at the time of publication. Sep 26, 2023 at 15:31

5 Answers 5

14

TLDR: It isn't.

Longer answer: Much medical research is flawed. How much is debatable, but it's a lot. But the flaws do not always (or even usually) make the research "useless" or "utter garbage".

For instance, in much medical research it is impossible to randomly assign people to conditions. E.g. you can't randomly assign people to "overweight" and "not overweight". That does not make the studies of obesity useless.

Medicine isn't physics or chemistry. People are complicated. Disease is (often) very complicated. There is a lot more error (in the statistical sense) in medicine than in physics. But that doesn't mean it is useless. (E.g. a lot of people who smoke don't get cancer, but so?)

Of course, there are ways medical research could be improved. I am a statistical editor at a couple of medical journals, so I do my small part to improve it.

Another issue is an over-reliance on randomized clinical trials, which reduce some kinds of error at the cost of increasing others. For one thing, people act differently when they are part of a trial than they do in real life. And RCTs often look at samples that have no other medical conditions when, in real life, people with one condition often have others. And there are problems with selecting too many White men and not enough women or people of color. And so on. It's by no means perfect. But there's a long way between "perfect" and "garbage".

For evidence, just look at the amazing progress medicine has made, much of it based on research.

Of course the "pressure to publish" doesn't help. But ... that pressure also applies to the people who are writing papers decrying the state of research.

6
  • I think a "TLDR" needs to be a bit longer than that
    – toby544
    Sep 26, 2023 at 10:45
  • I agree that medicine has made huge progress. I agree humans are complicated. But we still see thousands of new papers every day that answer the exact same question as thousands did before. Please read this nature.com/articles/533452a Sep 26, 2023 at 11:22
  • There has to be a main culprit here that is driving us away from discovery, towards publishing garbage. Sep 26, 2023 at 11:23
  • 3
    @AdamRobinsson - clearly we just have to find an easy algorithm that will predict which studies should not be done because they will be "useless" - should be simple, right?
    – Jon Custer
    Sep 26, 2023 at 13:10
  • Oops, RTC should be RCT, I will edit.
    – Peter Flom
    Sep 26, 2023 at 16:11
2

Apparently, there is a debate in the community, since you pointed out to Lancet "Reducing Waste". I guess they also point out to some solutions to problem, not simpy yelling at the produced waste.

The best path forward (best as "minimal effort and penalty for you, maximal long term gain) you can try is to keep on publishing your waste (as required by career, social pressure, contracts, research agreements and so on...), and continuously mention and refer to "Reducing Waste" of Lancet somewhere in your publication (I would suggest either the introduction or discussion section).

Advertising works very well that way, rewiring brains through repetitions, why should not this work in the medical community, too?

1

Why is 90% of medical research useless?

If that implies 10% of medical journal articles are useful, then I would say that's good performance. There are several fields of research that never produce anything "useful," some of which do not intend to be useful.

Publishing useless journal articles is useful in that it discourages other people from repeating the useless research (0% success rate). That encourages people to do something new (apparently 10% success rate). Some think publishing the useless results is obligatory.

5
  • 1
    0% success rate ? What are those fields ?
    – learner
    Sep 26, 2023 at 11:23
  • Thats not how medical research works. For EVERY research question that doesnt require industry funding, there are 1000s of papers doing the exact same thing instead of trying to discover something new, with the exceptions being a tiny fraction of the output in total. Sep 26, 2023 at 11:32
  • @ANG "useful" and "success" are not the same thing. Sep 26, 2023 at 14:34
  • 2
    I think you're addressing a different problem than the one OP means to, though they were pretty sloppy with their interpretation and citation in the original post. The research they have labeled "useless" based on the sources they refer to has more to do with p-hacking, underpowered studies, and publication bias, rather than usefulness in terms of practical utility. If anything, we're balanced way on the side of not repeating enough, while also not publishing the useful "useless" articles you refer to (like those with null results) that are less exciting but very important.
    – Bryan Krause
    Sep 26, 2023 at 20:22
  • @BryanKrause I don't think you should compare my answer with a citation the asker didn't put in the question. Also, I have now voted to close. Sep 26, 2023 at 22:08
1

If I simplify it, maybe one of the reasons that many of the doctors publish their papers based on data collection e.g., out of 100 selected patients, in certain age groups certain findings, and so on. Finally giving a conclusion at the end of the paper, which the doctor already knows. This is because many doctors prefer their knowledge-based profession instead of thinking analytically about some disease or ignoring critical thinking. Doctors are trained as practical professionals, not as researchers. They don't go through a PhD. They earn a huge amount of money by treating patients with existing diseases and become ignorant about unsolved problems (diseases in this case). To be precise, medical professionals, especially doctors, do their jobs on existing problems or existing diseases. So they are bound by the nature of their profession.

However, of course, there are some researchers in the medical sciences who don't practice as doctors and instead focus on finding solutions to unsolved diseases. Of course, those medical professionals publish pure, original research papers.

To conclude my answer, I would make a separation between medical doctors and medical researchers. The first category focuses on the treatment of patients and earning a flexible livelihood. While the second category is dedicated to real research to solve mysteries about many unsolved diseases, this second category certainly publishes fundamental research papers. Of course, researchers from biochemistry, bioengineering, applied mathematics, and biophysics also publish some original research works that are important for the medical sciences, as you can see among the Nobel Prize winners in chemistry in the last few years, some of who worked on destroying cancer cells, etc.

3
  • I agree that basic science researchers can have a larger failure rate. But the problem is that clinical research is more predictable so why does it not add more value? Sep 26, 2023 at 11:25
  • 1
    @AdamRobinsson, It is commonly believed that clinical research should have a lower failure rate than fundamental science research because it is more predictable. Nonetheless, a number of factors contribute to the complexity and unpredictability of clinical research, which can influence its perceived value and success rate such as "ethical consideratins", "sample size and statistical significance" etc
    – learner
    Sep 26, 2023 at 11:45
  • 1
    There's a lot of scientific sloppiness that goes on with research within the medical profession - a lot of that is attributable to the lack of training you mention. Another big chunk is simply money. Most physicians are not allocated sufficient time for research to run after grants and such. It's expensive to do solid clinical trials with sufficient power. It's relatively cheap to collect medical records data from your own institution (cost is really just your own time). As far as medical researchers, they're just as guilty of the sins of the field as the physicians...
    – Bryan Krause
    Sep 26, 2023 at 20:19
1

„Medical“ is a red herring. Any deep research, if done properly, is useless. The reason is that the term research primarily refers to creation, systematization, and maintenance of knowledge. It's not about being useful to anyone or for any purpose.

If you wish that what you do is useful, do something else. Don't do research. Or at least stop fooling yourself by expecting any kind of application of proper research. Or switch to applied research, which, by its definition, is meant to be applied in some area.

As for garbage in terms of poor knowledge, it's also not specific to medicine. Most of research out there is irreproducible or low-quality in some way. There are many reasons: poor publication standards (including inadequate control in publication), poor/inadequate/small empirical studies (including too much freedom for too few tested individuals), the contents (in some cases) being not interesting to anyone except the researcher and 1-2 other folks on Earth, etc. If you wish to do something higher-quality, go to an area with better control and higher interest in your product.

Not the answer you're looking for? Browse other questions tagged .