What to do when you spend several months working on an idea that fails in a masters thesis?

Question

How shall a masters student deal with the complete failure to meet the expected results when working on a master's thesis?

For example, in the field of machine learning a masters student might spend 4-5 months developing a method that turns out not be useful, not even being comparable to benchmark datasets.

Should the student quit it? Should you at least take a break from academia to avoid harming your career ? Or should you work on a different idea and risk another few months which is also not guaranteed? Given the fact that the advisor is simply asking you to try new things

Answer 1

Step 1: Don't panic

I was in a similar situation halfway through my MSc. I was in a panic, sure that my academic career was in ruins. My supervisor calmed me down, reminded that a negative result was still a result, and and told me that a for a master's degree, it was not strictly required that I make a scientific contribution or have a publication. In the worst case, in my thesis I would present my negative results, explain why this technique didn't work, and suggest what could be done differently by future researchers. (Once I was relaxed enough to think clearly, I came up with new things to try, and everything worked out grand.)

I suggest you discuss the "worst case scenario" with your supervisor; you'll probably find out it's not as bad as you think. Remember that this is research: positive results are not guaranteed.

Step 2: Think about why this technique isn't working.

I'm sure you've learned something about why your technique isn't working. That should give you some ideas for what to try next. If you're out of ideas, sit a friend down and explain everything to them. The friend doesn't need to know anything about machine learning; they're just a sounding board. The naive questions they ask may give you ideas. Maybe you need a week off to recharge your batteries.

Step 3: Try something new.

Take those new ideas you got in step 2, and apply them. But now that you're more experienced, think about how you could find out more quickly if the idea is feasible, so you can change tack again if needed.

Answer 2

Years ago Marguerite Lehr, a colleague of mine at Bryn Mawr College, told me of a conversation she'd had years before that with Oscar Zariski, a brilliant algebraic geometer then at Johns Hopkins. She told him about a failed attempt to solve a particular problem. He said "you must publish this." She asked why, since it had failed. He replied that it was a natural way to attack the problem and people should know that it wouldn't work.

Answer 3

Just a general answer, more to the overall issue than to your specific case: "Failure" to get the expected results isn't necessarily "failure" in the sense of not producing a good thesis. Although in the specific case you mentioned (machine learning) there is often a desire to produce something usable, in many cases a thesis topic is motivated by prior research. A negative finding can still be significant if it adds to the overall knowledge total in the area (for instance, by showing that predictions from earlier research are not confirmed by yours).

For my PhD, I spent over a year conducting a series of experiments to test a certain hypothesis derived from earlier research. I found no evidence in support of the hypothesis. Nonetheless, I wrote it up as a negative result, and framed it as placing limits on the theoretical proposals that motivated the project (i.e., "people suggested things might work like this, but I checked and apparently it's not so"). My committee thought it was a useful contribution and I got the PhD.

A lot depends on your field and your committee. It is easier to do what I described in a field where there is a lot of speculative theorizing relative to the amount of hard data. I can imagine it'd be a lot harder to do that in machine learning. Also, the bias against negative results (the so-called "file-drawer problem") can create pressure to produce a positive finding. In the broadest sense, though, if you had a good reason to go looking for something, not finding it can be as informative as finding it, and that's part of science.

Answer 4

I concur with the above answers. I spent a year working on a topic for a PhD dissertation, before I decided that I lacked the qualifications to do it. So I went to my advisor, and said, "What now?" Two weeks later I was starting the work that got me my PhD. Leaving academia will probably be a mistake; you may find that you never have the right time to go back. As pointed out, negative results can have value. I think of Gödel and the Completeness Theorem, wherein the concept of unsolvable problems was formalized. Instead of proving that first-order predicate calculus was complete, just the opposite was proven: it is possible to create theorems that can never be proven (see also Alan Turing, Church, Post, and for that matter, Heisenberg).

So take a look at your work. Proving that some idea cannot work is a contribution. And if not, try another topic. That's what advisors are for.