52

How shall a masters student deal with the complete failure to meet the expected results when working on a master's thesis?

For example, in the field of machine learning a masters student might spend 4-5 months developing a method that turns out not be useful, not even being comparable to benchmark datasets.

Should the student quit it? Should you at least take a break from academia to avoid harming your career ? Or should you work on a different idea and risk another few months which is also not guaranteed? Given the fact that the advisor is simply asking you to try new things

  • I think this presentation by Uri Alon can be relevant. It certainly helped me see things differently. – ddiez Nov 3 '14 at 12:10
  • 10
    Research is 95% failure and 3% success (the other 2% you don't have even a clue). Don't panic. – Ander Biguri Nov 3 '14 at 16:37
  • 6
    Research is only a failure if you fail to generate useful information. Knowing that an approach doesn't work is a useful result. – Murphy Nov 3 '14 at 17:35
82

Step 1: Don't panic

I was in a similar situation halfway through my MSc. I was in a panic, sure that my academic career was in ruins. My supervisor calmed me down, reminded that a negative result was still a result, and and told me that a for a master's degree, it was not strictly required that I make a scientific contribution or have a publication. In the worst case, in my thesis I would present my negative results, explain why this technique didn't work, and suggest what could be done differently by future researchers. (Once I was relaxed enough to think clearly, I came up with new things to try, and everything worked out grand.)

I suggest you discuss the "worst case scenario" with your supervisor; you'll probably find out it's not as bad as you think. Remember that this is research: positive results are not guaranteed.

Step 2: Think about why this technique isn't working.

I'm sure you've learned something about why your technique isn't working. That should give you some ideas for what to try next. If you're out of ideas, sit a friend down and explain everything to them. The friend doesn't need to know anything about machine learning; they're just a sounding board. The naive questions they ask may give you ideas. Maybe you need a week off to recharge your batteries.

Step 3: Try something new.

Take those new ideas you got in step 2, and apply them. But now that you're more experienced, think about how you could find out more quickly if the idea is feasible, so you can change tack again if needed.

  • 4
    +1 Exactly this. Failure happens and then you have to talk to your advisor. He will know what to do. – Alexandros Nov 1 '14 at 18:09
  • 10
    Very helpful advice. – Pete L. Clark Nov 1 '14 at 19:00
  • 8
    @EnthusiasticStudent a positive thesis ending in a published paper would be helpful; but a well written thesis that explains why the results are bad and what could be done about it, may be an equally good indicator of research capabilities. Remember, no one is expected to publish during the masters. – Davidmh Nov 2 '14 at 8:39
  • 5
    @EnthusiasticStudent I know many people with PhD funding and no publications before it. Me, for starters. I am sorry you haven't been able to find anything, but this is not necessarily the end. – Davidmh Nov 2 '14 at 12:58
  • 2
    @Magellanea I know cases in both. Even more, one of them rejected an offer from (I think it was) Harvard, because he got a better one. – Davidmh Nov 4 '14 at 9:57
28

Just a general answer, more to the overall issue than to your specific case: "Failure" to get the expected results isn't necessarily "failure" in the sense of not producing a good thesis. Although in the specific case you mentioned (machine learning) there is often a desire to produce something usable, in many cases a thesis topic is motivated by prior research. A negative finding can still be significant if it adds to the overall knowledge total in the area (for instance, by showing that predictions from earlier research are not confirmed by yours).

For my PhD, I spent over a year conducting a series of experiments to test a certain hypothesis derived from earlier research. I found no evidence in support of the hypothesis. Nonetheless, I wrote it up as a negative result, and framed it as placing limits on the theoretical proposals that motivated the project (i.e., "people suggested things might work like this, but I checked and apparently it's not so"). My committee thought it was a useful contribution and I got the PhD.

A lot depends on your field and your committee. It is easier to do what I described in a field where there is a lot of speculative theorizing relative to the amount of hard data. I can imagine it'd be a lot harder to do that in machine learning. Also, the bias against negative results (the so-called "file-drawer problem") can create pressure to produce a positive finding. In the broadest sense, though, if you had a good reason to go looking for something, not finding it can be as informative as finding it, and that's part of science.

17

Years ago Marguerite Lehr, a colleague of mine at Bryn Mawr college, told me of a conversation she'd had years before that with Oscar Zariski, a brilliant algebraic geometer then at Johns Hopkins. She told him about a failed attempt to solve a particular problem. He said "you must publish this." She asked why, since it had failed. He replied that it was a natural way to attack the problem and people should know that it wouldn't work.

5

I concur with the above answers. I spent a year working on a topic for a PhD dissertation, before I decided that I lacked the qualifications to do it. So I went to my advisor, and said, "What now?" Two weeks later I was starting the work that got me my PhD. Leaving academia will probably be a mistake; you may find that you never have the right time to go back. As pointed out, negative results can have value. I think of Gödel and the Completeness Theorem, wherein the concept of unsolvable problems was formalized. Instead of proving that first-order predicate calculus was complete, just the opposite was proven: it is possible to create theorems that can never be proven (see also Alan Turing, Church, Post, and for that matter, Heisenberg).

So take a look at your work. Proving that some idea cannot work is a contribution. And if not, try another topic. That's what advisors are for.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.