I am an MSc graduate and work as an independent researcher. I spent 10 months on a paper. I wrote many scripts in Matlab to implement four other comparative methods as well as a new method that my MSc professor had proposed. Each of these methods was so difficult to implement.

Unfortunately, the method that I was going to propose does not work well compared to the comparative methods, which were introduced already 3-4 years ago.

What should I do now?

  • 61
    This is how I spend most of my time. If you want to do research, you have to be prepared to be disappointed on regular basis. The state-of-the-art usually is what it is for good reasons. Many of your ideas won't work, or won't work as well as you hope. Research isn't like in the movies, where some savant improves an entire field just by having a quick glance over the problem. Commented Jun 23, 2015 at 7:07
  • 32
    It is of the utmost importance to make a fair comparison to prove state-of-the-art performance. If you cheat, you will get burned. You may get published, you may get cited, but if you get caught (which will happen when someone tries to repeat your experiments) you're in trouble. Integrity is extremely important in research. So in short: NO, that is not okay. Commented Jun 23, 2015 at 7:11
  • 19
    @Jamaisavenir no, it is not ok to report forged results. Commented Jun 23, 2015 at 7:17
  • 17
    @Greg Negative results are still worthwhile results. Chiding people for negative results does nothing worthwhile while encouraging research fraud and p-hacking. If we had more sensible/ethical reviewers and journals willing to publish negative results rather than always going for the "sexy" results then science as a whole would be vastly better off and the OP wouldn't need to treat this like a disaster.
    – Murphy
    Commented Jun 23, 2015 at 10:54
  • 11
    “I have not failed. I've just found 10,000 ways that won't work.” ― Thomas A. Edison Commented Jun 23, 2015 at 13:14

9 Answers 9


I recommend that you record your failures and not just your successes. In your case, I suggest you write a technical report giving all the details (the implementation, the comparison, the conclusion). It may not be considered as a publication worthy of your resumé, but at least it serves to document your efforts and may prove to be useful in the future (in case other people are thinking of using your proposed method).

Also note that your method may appear worse when compared to others using a certain metric (say, time complexity) but may appear better when compared to others using a different metric (say, space complexity). You might want to look at your method again.

  • 4
    i agree that a negative result is a result as well. also compare the answers to this thread: academia.stackexchange.com/questions/46300/… Commented Jun 23, 2015 at 7:20
  • 1
    In a previous edit OP said this is an algorithm of his research adviser. So, for political reason, he may not want to go around with a big flag stating it doesn't work and he proved it..
    – Greg
    Commented Jun 23, 2015 at 7:28
  • @Greg, thanks for pointing that out. To Jamais avenir, if the idea for the method came from your adviser, then you will need his/her permission before you write anything.
    – JRN
    Commented Jun 23, 2015 at 8:18
  • 1
    @Greg For other political reasons, the advisor might insist that the student wave a big flag stating that their method doesn't work and he proved it. In the long run, it's always better to admit your failures than to try to hide them.
    – JeffE
    Commented Jun 25, 2015 at 19:23
  • 1
    @JeffE In principle I agree, and I would recommend that to the SUPERVISOR. However, I don't think the OP should bet on it automatically, or go against the supervisor, if she/he do not share these principles.
    – Greg
    Commented Jun 26, 2015 at 1:10

In the past ten months, you achieved the following:

  1. Validate existing results and implement them in Matlab.
  2. Show the quality of an alternate method.

To me it seems clear that the only way to prevent this time from being a waste is to publish your work or conclusions. First of all, others who work with Matlab may be able to take your implementations and use them for further research. Secondly, if you don’t share your findings on the alternate method, someone else may decide to spend ten months on it before coming to the same conclusion.

That should be sufficient motivation, but of course your negative result can be considered positive in an unexpected way. Either right now by you, or later by someone else.

  • 2
    +1 for "if you don’t share your findings on the alternate method, someone else may decide to spend ten months on it." This is why the research was not a waste of time – even if it didn't find the breakthrough that was initially hoped for.
    – J.R.
    Commented Jun 26, 2015 at 8:25

Next time, perhaps implement only one of the already existing methods and then the newly proposed idea. If the litmus test is that your new idea outperforms all of the baselines, you can test that already early on.

In general, incrementality is key: if you're looking at a task that is going to take you 10 month to realize you need to be aware of the risk you're taking. Be prepared that the results may not end up being what you hope for, or, if you cannot take that risk, find an alternative way of moving forward. Good planning is a valuable skill to have.

For now, would it be possible to analyse in depth why the new method fell short? Is there perhaps a certain sub-problem that it does very well on, in which case you could still extract a (somewhat weaker but) positive result?

Minimally, I'm sure you gained some new insights, and perhaps are even in a position now where you can correct some misconceptions you might have had about your problem: the outcome of your experiment is that something inherent in your problem is not like you thought it was, or else your method would have worked. So you may ask yourself what is it that's different?

If an unexpected obstacle drops on your path, don't just stop. Look for a way around it, or a new direction altogether.

  • +1 for the incrementality suggestion and for the suggestion to analyze why the new method fell short.
    – Wayne
    Commented Jun 24, 2015 at 15:32

The first step is to see if your research is salvageable.

Does your method offer any advantages? Does it work better in certain scenarios? Does it require less/different inputs, and so will be useful where only those inputs are available?

Can you imagine any scenario where it WOULD be better to use your method over the state of the art?

While it can be difficult to publish null findings (i.e. my method wasn't better), it is generally possible to publish findings which include some form of positive results (i.e. my method wasn't better overall, but was better when x).


Publishing the results of research allows others to assess these results and build upon them. If the results are negative, it is still worthy for science to publish them because this will prevent others to spend precious time by trying to do the same and failing, without knowing that somebody else has already done it. Negative results may be published without review on repositories such as Arxiv or Zenodo (the later gives a DOI that may improve citability) or, with review, in journals such as PeerJ or F1000 Research (the later even had once a promotion for negative results).

Some may frown upon publishing in relatively low-ranked venues (where negative results would typically find their place), and a reason may be the opportunity cost of spending time to publish the result. I would argue that the time spent for publishing the result is minimal relative to the time spent doing the actual research that led to the negative result. If the time spent for publishing the result would prevent somebody else to spend an order of magnitude more time trying independently the same thing and failing, then publishing would bring an overall positive value to science, and therefore is worth doing.


These aren't poor results. Assuming your methods are good, they are good results - they answer a question. They may not be the results you want, but they're still good.

If you don't publish somewhere findable then someone else is going to waste their time unknowingly repeating work you've already done.



Many sub-areas of computer science not only value novel algorithms, but also case studies. While the top conferences and journals tend to favor algorithmic contributions, your can still go for the not-excellect-but-good conferences with a good case study.

So if you can extend your experimental results to a good case study, this is still a good contribution. Often, this mean crafting the benchmarks in a much more careful way than you would do it for showing that your novel technique is good, as you need to convince the reader that the cases that you consider resemble the practice well. So you may need to invest some more time.


Well research is all about finding new things and not getting the desired results. Since you planned to achieve something and working on that in the course of your tasks you must have found some undesired results. Its better to focus on what you got and what caused that instead of focusing on what you wanted to get. Hopefully you will reach your goal.


As a researcher all results are Great results, especially when the outcomes are unexpected or under expectations. It is these moments of 'failure' that the greatest discoveries of our times have come from, it is the exception to the 'rule' that have resulted in whole new disciplines and industries. there are no poor results, only poor analysis of the results. Case in point the Michelson-Morley experiment that had completely unexpected results that led to the Special theory of Relativity, and Pauli's discovery of the neutrino because of a mass loss.

  • This might be consoling, but it does not provide an actionable answer to the question. Can you explain how your remarks might help the OP to figure out what to do with their project? Commented Jun 23, 2015 at 17:16
  • 1
    All results are great results? Really. What if OP failed to prove proposed method are significantly different than comparative method. A priori we know one method is better than the other. We just don't know which. In that case, OP's tests were not sensitive enough.
    – emory
    Commented Jun 24, 2015 at 19:18
  • thus proving a result is a great result, in this case the need to find a better differentiating parameter. in other words the test needs to be refined further. or the proposed method is non-sequitor to the problem at issue
    – SkipBerne
    Commented Jun 24, 2015 at 19:23
  • You've provided evidence that some failures are great results. That's quite far from your claim that all results are great results. It's just not true; 90% of all results are crud (but then, to plagiarize Sturgeon's Law, 90% of everything is crud).
    – JeffE
    Commented Jun 25, 2015 at 19:19
  • 1
    and this site is proof.
    – SkipBerne
    Commented Jun 25, 2015 at 19:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .