Is a research thesis (report) with zero contribution to human knowledge acceptable?

Question

An alternate title could be "Do we award a PhD for what the student became or for what he produced?". More precisely, if a research thesis report does not advance human knowledge, yet still shows that the student acquired the right skills of investigation, should the title of Philosophy Doctor be awarded?

• I kind of always assumed that a thesis (PhD thesis in particular) should have some positive results (where positive means "advancing the state of human knowledge"), and that part of the art of finding a good topic of research was the art of asking the right kind of question, which would yield some positive result independently of the answer. Similar views are expressed in this other stackexchange question and the corresponding answers:

  It is an expectation that the PhD would make an original contribution and/or advance knowledge in a given field. I understand this is a universal assumption for this level of study across all universities. (...) usually a PhD is measured on its contribution to expand knowledge.

• Nevertheless, Justin Zobel defends convincingly the opposite view in "Writing for Computer Science", p.154 of the Second Edition:

  even if good results are not achieved, the thesis should pass if you have shown the ability to undertake high-quality research. (...) A thesis with negative results can, if appropriately written, demonstrate the ability of the candidate just as well a a thesis with positive results. (...) it is you, not the research, that is the primary object of scrutiny

Is there an agreement across disciplines about this question?

I am not directly concerned (I reported positive results in my PhD thesis long ago, have many positive results to report in my "Habilitation" thesis, and I certainly aim for my students to report positive results in theirs), but I am curious about the real objective of the thesis: - as an advisor, I could suggest a more risky topic if it had the potential to teach more to the student without risking the whole graduating thing; and - as a referee or member of an evaluation committee, I have to judge students and/or their thesis...

---

Extreme Fictional Example

A student and advisor do the entire research work following the most rigorous scientific process for several years, only to find their efforts ruined near the end of the process either by a budget cut, the disappearance of the species they were studying, or the discovery that the problem is the consequence of an obscure results from year ago in another research community.

The student has followed and learned the scientific process, but did not contribute to human knowledge (apart from maybe improving the index of its bibliography). If the student has showed the qualities required from a good researcher, should(n't) he/she be awarded the title of "Doctor in Philosophy", independently of the contribution made to human knowledge?

This is truly a rhetorical question, and I doubt this kind of situation happens often. Yet the idea is new to me and I kind of like it, albeit I doubt the whole community would agree...

---

Opposite Extreme Example

Imagine that a student, stroke by luck, makes an amazing scientific discovery which deeply impact human knowledge, and can be understood by all even though the student poorly redacts it. It seems clear to me that the society would not benefit from awarding a PhD to such a student, who has not learned how to do research even though contributing to human knowledge.

On the other hand, setting two conditions for the awarding of a PhD, having learned how to do proper research AND having advanced human knowledge by using it, introduces trade-offs and compromises (which again do not serve society).

Answer 1

The question as currently asked is: Is a research thesis (report) with zero contribution to human knowledge acceptable?

And the answer to that question is no.

A thesis or portfolio submitted for a PhD or higher doctorate must make a novel contribution to human knowledge. It must also demonstrate that the applicant has acquired the appropriate level of research skills.

Some negative results do advance human knowledge. So a thesis with negative results and no positive results may make a novel contribution to human knowledge. e.g. demonstration of absence of an effect is a negative result, but can be a distinct and significant contribution to new knowledge (particularly if the effect was previously believed to exist).

However, just spending the time, putting in the effort, and churning out the right quantity of work, is not in and of itself sufficient.

Basis for this

This is based on a combination of my employer's guidelines, my experiences as a PhD supervisor, and advice from my colleagues. I hear that there are other (less well-respected) institutions that award doctorates just for putting in the effort and churning out the right quantity of work, regardless of novelty of contribution, or of demonstration of research skill

A quote from some official guidelines.

Here's a quote from the relevant part of the academic regulations for PhD examinations from UCL, University College London (pdf):

A thesis for the awards of EngD or PhD degree shall be examined in accordance with the criteria prescribed by UCL and the thesis shall demonstrate that it: ...

shows a student's capacity to pursue original research in the field of study based on a good understanding of the research techniques and concepts appropriate to the discipline; ...

represents a distinct and significant contribution to the subject, whether through the discovery of new knowledge, the connection of previously unrelated facts, the development of new theory, or the revision of older views;

Answer 2

I assume that by negative results, you mean non-significant results.

"It is an expectation that the PhD would make an original contribution and/or advance knowledge in a given field." Yes, this is true. And "X doesn't work" is a contribution to the field.

(To use an example from my area). Health practitioners are constantly dreaming up things that might work to treat various ailments (illnessess), and using them. The job of health researchers is to find out which ones work - and most of them don't work.

We used to joke that our role as health care researchers was to say no. "Nope. That doesn't work. Don't do it. No, that one's not effective either. No, don't use that. No. No. No."

Pressure for positive results just means you tweak models and data until you find them - using 'researcher degrees of freedom' (see http://pss.sagepub.com/content/22/11/1359). Your results are therefore significant, but worthless.

My PhD thesis was trying to demonstrate the nature of the relationship between stress and psoriasis symptoms (many people say "stress worsens psoriasis" - it's taken as a given truth, but it's never been empirically demonstrated). I was trying to answer things like what kind of stress, how long does it take, does it differ between people? I never found any evidence that stress did worsen psoriasis. Nor that psoriasis worsened stress (or any other psychological symptom).

A PhD is an educational process. One should demonstrate that one has learned. The most important thing about a PhD is showing what you know, what you have learned, and what you understand. If anyone gets to the end of a PhD and says "Well, those results were all positive, just as I expected", they've learned little. At the end of your PhD (or any research project) you should want to start again, and this time do it properly.

Answer 3

This is a tough question. Just to abstain from the discussion of different standards in different fields, I'll talk of mathematics only. Also, I'll assume that it is a question about a PhD thesis, not about anything of lower level like Masters, etc.

The main thing is that the gap between "advancing human knowledge" and "mastering the subject" is huge and there is a lot of grades in-between. IMHO, the works that advance human knowledge are rare, be it PhD theses or papers in refereed journals. Most of us live off "doing what hasn't been done before", which is much less demanding. What I mean is that each work introducing a new idea is followed by 1000 ones applying this idea combined with already known stuff to all setups where it works. Each of those 1000 papers does what hasn't been done before but does not advance human knowledge because, once the new idea appears, every sufficiently high level professional can figure out how it may be applied elsewhere, though getting all details right may require patience and even some effort. I certainly would accept "doing what hasn't been done before" (a successful application of a well-known idea in a fairly straightforward way to a new setup) as a tolerable (but not brilliant) PhD thesis.

How much below that would I consider acceptable? Four out of every five projects I try end up in a miserable failure, when I cannot even claim that I have proved some partial result in the desired direction. I have never tried to write a detailed account of "mein kampf" for any of those (dead end moves with counterexamples at the end, chains of implications that never meet the goal, associations and studies of seemingly relevant things that failed to relate to the question at hand for some fundamental but hard to discern reason, etc.) but I would let a report like that pass as a PhD thesis if it really shows 3 years worth of high quality effort.

What I find not acceptable is a "literature survey" (understanding what is written elsewhere and relating things in a superficial way without deriving any new result or introducing any new twist into the story). In other words, my idea is that you should get your PhD after you show that you can "fight a mathematical battle on your own", not only study the battles fought by other people. The victory in a decent battle is sufficient, but not necessary. Sometimes you can be made a "general" even if you lose but show good fighting skills.

All this is my humble opinion only. As to the official point of view, in Russia we had the central committee that had to confirm every degree award before it became valid and there were written guidelines. In the USA it is way more relaxed, so 4-5 professors conspiring together can pass anyone (to the credit of them I should say that I cannot give an example of such conspiracy). Canada requires an external review to be positive (which, by the way, makes perfect sense as a simple safeguard against "local standard relaxation" to me), and so on.

As to "risky topics", the best ambitious projects are such where "something" can be done right away (not something that is worth talking about as "defendable" or "publishable", but something that shows that the student has the general grasp of the subject and decent problem-solving skills). If that something (or something equivalent) is not done within the first half a year, it is a sign of trouble and the ambitious project is better abandoned and replaced with an "apply a known idea in a straightforward way to a new setup" one. If it is, you have a chance and may consider taking the risk. Unfortunately, there is nothing that can guide you then except your gut feelings and your knowledge of the student. You and your student are in an uncharted territory all on your own, and no general advice can be given except "play by ear" and "act by circumstances".

Answer 4

Well, this negative result got a fair amount of press, this negative result is generally considered a big deal, and my guess is that a negative result about this problem would probably be considered an acceptable thesis. :-) So the trivial answer to the original question is: "Yes." On the other hand, it's easy to think of negative results that wouldn't pass muster for a thesis. So perhaps the underlying question is: "How can we tell whether a negative result qualifies for a thesis?"

A thesis is expected to make an intellectual contribution. If I prove a bunch of trivial negative results that surprise no one, then I don't make any intellectual contribution. On the other hand, if a lot of good researchers believe X, and I show that X doesn't hold, then that changes the state of knowledge in the field and therefore is a contribution. (In many cases, the real contribution from such negative results is the analysis explaining why X doesn't hold --- i.e., why the scientific intuitions of a bunch of good researchers are incorrect.)

So the questions I would ask would be: are the negative results unexpected? Do they give us new insights into, or a better understanding of, the phenomena being studied? If these questions can be answered positively then I think the negative results qualify for a thesis; if not, then IMHO they do not. The bottom line is, simply: "Do the results (positive or negative) teach us anything?"

Answer 5

In fast moving fields one can get scooped fairly easily. For example a biology PhD thesis may be based around determining the structure of a protein. If someone else publishes the structure before the thesis is reviewed then there is not a contribution to human knowledge since the structure is already known. I think in these fast moving fields the student would be expected to do more. In my "slow" moving field I am aware of two theses (one PhD and one MSc) where the results where the key findings were published by someone else in the weeks before the thesis was finished.

Answer 6

An historical example: Alan Turing was named a fellow at Cambridge on the basis of work that Lindeberg published over a decade prior. Admittedly, this was not a PhD thesis, but I am under the impression the import is comparable. Keynes, for example, had no PhD but was named a fellow prior to becoming a professor. There is no evidence to suggest that Turing was aware of Lindeberg's work, but I am under the impression that they're approaches were quite similar.

I have heard similar stories on a less grand scale. No doubt the results of many theses have been previously published. Should it matter whether this connection is discovered before or after the thesis is submitted, assuming the new results did not draw on the old?

Answer 7

For a research project to be successful, you need more than just

• a good student

You also need

• a good concept and
• a good advisor

Of course these points are not independent: up to a certain point a good student will be able to make sure the concept is good. At least the good advisor will do that. But what if the advisor isn't that good, and in consequence the project concept is flawed and the student gets bad advise and that is the reason for only negative results?

---

IMHO, having only negative results likely hamper the project success in an indirect way:

• do not underestimate the psychological effect on the student: not succeeding in something that the advisor told you to do can have serious effects on the student's self-esteem and that can in turn lead to giving up or anyways loose the "psychological bonus" that you can only get if you are convinced of your work.
• Depending on what kind of negative results, and how the project was specified, "it doesn't work" could be either due to the student's inability/laziness or due to the fact that things just don't work that way. Therefore "it doesn't work" always has a danger of falling back onto the student, even if it was not the student's fault.
• It is hard for the student to prove that it wasn't his fault in that case, which may mean wasting lots of effort just to make sure that noone can come and blame the student for the bad results.

---

All together, I'd be extremely cautious with

I could suggest a more risky topic if it had the potential to teach more to the student without risking the whole graduating thing

Whether I'd at all consider this would also depend on other circumstances. Maybe it is because I've seen bad advisors putting students into risky projects and leaving them in the resulting difficulties.

So at the very least I think that entering a project that the advisor already before it ever began perceives as more risky, should be entered

• by a clear understanding between advisor and student that this is risky (but much may come out of it)
• only if the advisor knows the student well enough to be able to judge whether the student could stand this
• only with a clear risk-management strategy: the risk the student takes must be made up by something else.
  How about treating such risky projects as normal job and paying a full wage if you cannot guarantee that even a good student will be able to get a thesis out of the results?

Here's an example (heard of it by rumour only) of how such things go wrong:

project involves growing some plants over 2 years. If all goes well, student can finish thesis in 3 years.
Student is hired for the project on a fixed 3 year contract, and is told that the project really shouldn't take longer. (Apparently, there is no money for more than 3 years). In the 2nd year a minor flooding kills the experiment. Floods are in no way an unheard of occurrence in the region.

IMHO in this project the student was left not only with a project that had the inherent risk that the work of 2 years may be lost just before it was "ripe". The really bad thing is that it was combined with no plan B (e.g. extending the project), and occured in a country where a failed thesis is a serious thing: it wouldn't work to say in the beginning: "let's try this, and if it fails after 3 years, why, then you can just go on and try another project".