Pros and Cons of including less interesting results

Question

For the problem I am trying to solve, I considered 2 base algorithms and devised about 30 variations on each.

I then threw about 2 CPU-years and ran a number of simulations to see how they went.

When that completed I drew alot of graphs and came to some conclusions like:

1. Variations of the form X do almost exactly as well as the control
2. Variations of the Y family can be predicted to have unpredictable (and thus useless) results
3. The d[1,0*] variation is great under these conditions
4. The d[0*] variation is great under these other conditions

I took the last 2 and made a new algorithm and then tested that and found further useful results.

Now I am going to write my findings up into a paper.

I have 3 kind of results above:

• Points 3,4 (and the subsequent improvements) are interesting and will be the main focus of the paper.
• Points 1, 2, are kind of nonresults. They are failures, they did not produce anything useful.
  • For most it isn't even surprising that they didn't.
  • For others they are a approach by taken on a similar problem in a paper that inspired me to try and solve this related problem.

So should I comment at all about algorithm variations that were tested and found to not be good?

Why/why not?

Pros I can see:

This would help prevent others, not spend time trying them. I have read that it is a problem that in many disciplines (including this one) "negative results are not published".

Cons I can see:

Takes up space, may confuse reader as to which algorithm is the focus of the paper.

It feels abit weird that of the 2 CPU-years I spent testing these, and the considerable time I spent making the tests, I will only tell the rest of the world about 5% of my results.

Answer 1

You could omit the uninteresting 95% in the peer-review journal version, but include them as an appendix in the arxiv.org version of the paper. That way they're available but you're not forcing anyone to read them...

Answer 2

Negative results are as interesting, in my opinion, as positive ones. They allow future researches to know what won't work, and prevent them wasting their time (unless they thing you did something wrong- in which case they will try it anyway :-). However, the main paper should probably focus on the positive part, since most researchers have enough to read already and will not care much about the negative results.

In biology at least, it is frequent to mention some negative result and then include "(not shown)". But nowadays with the facilities to add supplementary material there is no reason to not include some proof. I would suggest to include negative results as supplementary material and just mention them in the main manuscript.