I am an MSc graduate and work as an independent researcher. I spent 10 months on a paper. I wrote many scripts in Matlab to implement four other comparative methods as well as a new method that my MSc professor had proposed. Each of these methods was so difficult to implement.
Unfortunately, the method that I was going to propose does not work well compared to the comparative methods, which were introduced already 3-4 years ago.
What should I do now?
I recommend that you record your failures and not just your successes. In your case, I suggest you write a technical report giving all the details (the implementation, the comparison, the conclusion). It may not be considered as a publication worthy of your resumé, but at least it serves to document your efforts and may prove to be useful in the future (in case other people are thinking of using your proposed method).
Also note that your method may appear worse when compared to others using a certain metric (say, time complexity) but may appear better when compared to others using a different metric (say, space complexity). You might want to look at your method again.
In the past ten months, you achieved the following:
To me it seems clear that the only way to prevent this time from being a waste is to publish your work or conclusions. First of all, others who work with Matlab may be able to take your implementations and use them for further research. Secondly, if you don’t share your findings on the alternate method, someone else may decide to spend ten months on it before coming to the same conclusion.
That should be sufficient motivation, but of course your negative result can be considered positive in an unexpected way. Either right now by you, or later by someone else.
Next time, perhaps implement only one of the already existing methods and then the newly proposed idea. If the litmus test is that your new idea outperforms all of the baselines, you can test that already early on.
In general, incrementality is key: if you're looking at a task that is going to take you 10 month to realize you need to be aware of the risk you're taking. Be prepared that the results may not end up being what you hope for, or, if you cannot take that risk, find an alternative way of moving forward. Good planning is a valuable skill to have.
For now, would it be possible to analyse in depth why the new method fell short? Is there perhaps a certain sub-problem that it does very well on, in which case you could still extract a (somewhat weaker but) positive result?
Minimally, I'm sure you gained some new insights, and perhaps are even in a position now where you can correct some misconceptions you might have had about your problem: the outcome of your experiment is that something inherent in your problem is not like you thought it was, or else your method would have worked. So you may ask yourself what is it that's different?
If an unexpected obstacle drops on your path, don't just stop. Look for a way around it, or a new direction altogether.
The first step is to see if your research is salvageable.
Does your method offer any advantages? Does it work better in certain scenarios? Does it require less/different inputs, and so will be useful where only those inputs are available?
Can you imagine any scenario where it WOULD be better to use your method over the state of the art?
While it can be difficult to publish null findings (i.e. my method wasn't better), it is generally possible to publish findings which include some form of positive results (i.e. my method wasn't better overall, but was better when x).
Publishing the results of research allows others to assess these results and build upon them. If the results are negative, it is still worthy for science to publish them because this will prevent others to spend precious time by trying to do the same and failing, without knowing that somebody else has already done it. Negative results may be published without review on repositories such as Arxiv or Zenodo (the later gives a DOI that may improve citability) or, with review, in journals such as PeerJ or F1000 Research (the later even had once a promotion for negative results).
Some may frown upon publishing in relatively low-ranked venues (where negative results would typically find their place), and a reason may be the opportunity cost of spending time to publish the result. I would argue that the time spent for publishing the result is minimal relative to the time spent doing the actual research that led to the negative result. If the time spent for publishing the result would prevent somebody else to spend an order of magnitude more time trying independently the same thing and failing, then publishing would bring an overall positive value to science, and therefore is worth doing.
These aren't poor results. Assuming your methods are good, they are good results - they answer a question. They may not be the results you want, but they're still good.
If you don't publish somewhere findable then someone else is going to waste their time unknowingly repeating work you've already done.
Publish
Many sub-areas of computer science not only value novel algorithms, but also case studies. While the top conferences and journals tend to favor algorithmic contributions, your can still go for the not-excellect-but-good conferences with a good case study.
So if you can extend your experimental results to a good case study, this is still a good contribution. Often, this mean crafting the benchmarks in a much more careful way than you would do it for showing that your novel technique is good, as you need to convince the reader that the cases that you consider resemble the practice well. So you may need to invest some more time.
Well research is all about finding new things and not getting the desired results. Since you planned to achieve something and working on that in the course of your tasks you must have found some undesired results. Its better to focus on what you got and what caused that instead of focusing on what you wanted to get. Hopefully you will reach your goal.
As a researcher all results are Great results, especially when the outcomes are unexpected or under expectations. It is these moments of 'failure' that the greatest discoveries of our times have come from, it is the exception to the 'rule' that have resulted in whole new disciplines and industries. there are no poor results, only poor analysis of the results. Case in point the Michelson-Morley experiment that had completely unexpected results that led to the Special theory of Relativity, and Pauli's discovery of the neutrino because of a mass loss.
