63

Typically, PhD theses are expected to have novel results. As science advances, it is becoming increasingly difficult to accumulate enough knowledge to begin a project that pushes the boundaries. Original findings are often published as papers before the defense, thus the thesis has become valued for its indication of technical expertise, not the discovery itself.

Since ancient history, redoing the work of masters has been seen as a very effective tool for training. It so happens that there is also a need for reproduction studies, but few are willing to do them.

Why don't PhD theses consist of reproducing important, recent, controversial research? Has anyone tried this?

Note: I am deliberately leaving open to interpretation whether the student would reproduce their own advisor's work or that of a researcher from a different group, and whether the original researcher is to assist in this and to what extent.

Improve this question
4
  • 20
    "As science advances, it is becoming increasingly difficult to accumulate enough knowledge to begin a project that pushes the boundaries." [citation needed]
    – David Richerby
    Commented Sep 18, 2016 at 18:06
  • 1
    I would prefer to advocate for a complementary path that emphasizes reproduction studies and have it not culminate as a PhD. It should be a valued career with appropriate degree.
    – user2338816
    Commented Sep 18, 2016 at 21:40
  • 3
    Completing a PhD means you are a self-sufficient scientist. The fact that you are able to reproduce somebody else's research does not prove it. Unless your area of research is making research reproductive, but I think that's not what you are asking about.
    – nuoritoveri
    Commented Sep 19, 2016 at 9:29
  • "novel results" is a very badly defined standard. To get a doctorate, you need to come up with a scientific question, a hypothesis, and answer it. A hypothesis can be a well educated guess at a possible flaw in a previous study, why not? A problem only arises if your original question proves to be wrong, i.e. what you checked was not actually a weak spot.
    – Karl
    Commented Sep 19, 2016 at 18:28

5 Answers 5

Reset to default
57

A chapter that does that is fine; I had one in my thesis. It can for example serve to set the stage for your original work. But someone with a PhD has to be able to do more than just repeat the work done by others, and the thesis has to show that capability. Replications are valuable, but a thesis consisting solely of replications won't show what it needs to show.

Instead we should try to find a way to make replications publishable (especially if they show what is already known) and worthwhile for established authors, rather than diminishing its value by delegating it to students.

Improve this answer
2
  • 8
    This surely depends on the context. Asking a student to reproduce Newtonian gravitation wouldn't contribute anything of substance. The theory is very well-founded. However, if there is a disputed or controversial result, like the discovery of an alien signal, then replicating that with a different setup would surely be of worth.
    – Bruce Becker
    Commented Sep 17, 2016 at 17:51
  • 4
    I mentioned in my answer that replication is often worthwhile, but that does not mean that a dissertation consisting solely of replications is right. The purpose of a dissertation is to show that the student can be an independent researcher. For that the dissertation should contain some original research.
    – Maarten Buis
    Commented Sep 17, 2016 at 18:58
33

Yes, "redoing the work of masters has been seen as a very effective tool for training." but the Ph.D. dissertation is not training! It evidence of what you have learned from the training.

Improve this answer
6
  • 8
    But Bachelor and Master thesis are training. Should we do reproduction in those?
    – Sumyrda - remember Monica
    Commented Sep 17, 2016 at 19:56
  • 6
    @Sumyrda - You could ask this as a Question (re Masters).
    – aparente001
    Commented Sep 17, 2016 at 21:12
  • 4
    @Sumyrda any lab course in any subject does this.
    – user18072
    Commented Sep 18, 2016 at 4:29
  • 8
    @Sumyrda: "But Bachelor and Master thesis are training." [citation needed]
    – O. R. Mapper
    Commented Sep 18, 2016 at 6:59
  • @Sumyrda I think the exact same answer would apply for Bachelor/Master theses as well.
    – luator
    Commented Sep 19, 2016 at 13:55
10

Reproducing and repeating results are not the same thing. Reproducing a previously-published result definitely a valid means of knowledge production, and so would be fine for a Ph.D. However, it should follow the scientific method. In particular, the new contribution should use a different method, or environment. Merely repeating the experiment or work can show that the method is self-consistent, but reproducing the result in a different context is one of the key tenets of the scientific method.

Improve this answer
2
  • 7
    Even plain repeating an experiment is worthwhile, and good practice, as the psychologists are now starting to find out. However, the question is: should we allow for dissertations that consist solely of replications. When we define replications narrowly, then my answer would be a firm no, even though they acan be useful, the dissertation does not show what it needs to show. If you broaden the definition, as you propose, then I see more room.
    – Maarten Buis
    Commented Sep 17, 2016 at 19:08
  • 2
    @Maarten Not only psychologists are finding that one out. If anything psychology is ahead of most other fields, where replication studies are just as rarely done, non prestigious and hard to publish. If we did similar experiments in biology or physics I doubt those filelds would fare any better (as a computer scientist I'd honestly put my money on computational biology "winning"). Still I agree that a PhD thesis should be original research. There have been calls for requiring PhD students to do at least one reproduction study though
    – Voo
    Commented Sep 18, 2016 at 18:28
5

I agree with all the other answers that a PhD thesis should still be original research. Still it is obvious that science in general (although clearly psychology is in the spotlight at the moment) has a problem with reproducing results.

This shouldn't surprise anyone - the process is set up in such a way that reproduction studies are hard to publish in good journals and non prestigious even if you get them published. All incentives are for researchers to not waste time on such studies but try to get novel results.

There have been a number of proposals on how to solve this problem, but one proposal in particular seems similar to your idea:

in order to receive a Ph.D. in psychology from any accredited institution in the United States (and perhaps in other nations as well), it is a requirement that one will have (1) conducted a high-quality “direct” replication of a major finding in their area (i.e., the area upon which their original doctoral research will be based); (2) written up the replication attempt to professional standards, no matter which way the data come out, and (3) made a good-faith effort to publish the paper in one of a growing number of high-quality online journals (such as PLoS ONE) that publish reports of well-conducted, valid experiments regardless of their novelty or their perceived “importance.”

A tragedy of the (academic) commons: interpreting the replication crisis in psychology as a social dilemma for early-career researchers

So you'd still do your novel research for your PhD thesis but be required to also conduct a high quality replication of some work in your field.

Improve this answer
9
  • Replicating a study only shows that the original study wasn't a fluke or fake. That's police-work. ;-) Reproducing the result (or refuting it!) after varying certain parameters you have identified to be possibly critical is totally science.
    – Karl
    Commented Sep 19, 2016 at 18:14
  • 1
    @Karl You're making my point. What you rather dismissingly (hopefully unintentional) call "police work" is at the core of the scientific method (which makes it "totally science" by definition). It's one of the pillars of science and absolutely undervalued and the whole scientific community is suffering because of this. Yes expanding and altering a hypothesis is important (and oh so much more prestigious and interesting), but if you don't make sure that your assumptions are actually valid you're building houses on sand.
    – Voo
    Commented Sep 19, 2016 at 20:57
  • Don't get me wrong, police work is very honourable. But if you go and repeat a study 1:1, without working out your doubts and misgivings and taking them into account while doing the actual work, then you're a police officer, not a scientist. Or neither, actually: The police officer would restrict himself to looking only into the suspicious parts. ;-) Or, putting it another way: If the original paper looked flawless and is totally convincing to you, would you be tempted repeat it? You'd just fall into the same traps as the original authors and never find the error, if there was one.
    – Karl
    Commented Sep 19, 2016 at 21:37
  • @Karl Indeed, it's essential for the proper working of the scientific method to redo that perfectly fine looking study. Look into the reproduction study done in psychology which was all over the news. Many of the results that couldn't be reproduced were published in top scientific journals, peer reviewed and looked "flawless" to others for years. There are so many ways that empirical studies can be distorted that there is just no replacement for other people reproducing the results.
    – Voo
    Commented Sep 19, 2016 at 21:47
  • That something which is essential to the basic principle of science is regarded so lowly ("not a scientist, just a police officer") is sad and has horrible consequences for the whole field. We've already seen the consequences in psychology and to a lesser degree in biology, but that's just because nobody has really looked into other areas.
    – Voo
    Commented Sep 19, 2016 at 21:49
0

While I agree with @Superbest on the hardship of producing novel results on the boundary of science, this issue satisfies for most of the things in the world now. Indeed, the novelty in the cinema and movies is very hard due to the production of hundreds of movies in the world just in one year.

But, how we can response to this problem? Just stop, redo and reproduce the old things again and again?

The pure original ideas appear scarcely in the academics and cinema (e.g.). But the hybrid ideas would be built from the pioneers of the past. The most published works (or even produced movies) are hybrid and not original ones'. The reproducing published works is an essential step for creating novel and potential ideas from the deficiencies of those works.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .