60

I wonder if is it okay for a researcher to suggest a new technique in a field of research and prove that this new suggested technique is a bad suggestion and gives no or negative gain and publish his work as a research on bad idea? (I'm in computer science but my question is a general question about every other field.)

Of course if there is some believe and doubt in the field about the idea, the researcher can prove it is correct or not in a research and it would be totally a productive result. However, if the idea is totally new and he just wants to prove incorrectness for future researchers, is this type of research acceptable in academia?

If not, why?
What is wrong with proving some never-existing idea is not a good idea?

I read this and I want to know what the current academical approach about such a research result is and the reasons for it.

Improve this question
9

11 Answers 11

Reset to default
69

What is wrong with proving some never existing idea is not a good idea?

Ideas are easy to produce, but unfortunately only a tiny fraction of them are really useful. If you pick an idea at random, it's likely to be not a good idea, and in most cases it would be relatively easy to check it and see it. So, your referees and your editor may wonder why is it so important to publish this random negative result, particularly in competition with other ideas, which actually lead to improvements and positive results.

why-science-needs-to-publish-negative-results

Some negative results are indeed worth publishing. For example, when you consider not a random idea, but a mainstream direction, which is believed to be superior all the time, and demonstrate that for a particular class of problems, or in particular setting, it does not work. So, your negative result essentially is an important warning that some popular and blindly trusted method is not yet well understood and should not be used as a silver bullet. Publishing such result can help preventing serious mistakes and hopefully start important discussion, eventually leading to improvement (or ban) of existing methodology.

Even in this case, you will need to do a good job convincing your referees and editor that this negative result is worth publishing, particularly if they are stand behind the idea you prove to be not good. Academia is very much influenced by the modern culture of success. It's much easier to become successful in academia when you talk about your success (however marginal and simple), rather when you produce negative results (however important and difficult).

Improve this answer
6
  • 20
    One famous such example in computer science is the 1972 paper by Klee and Minty proving that the simplex algorithm is not worst case poly time.
    – DreamConspiracy
    Sep 20, 2018 at 17:30
  • 5
    “you will need to do a good job convincing your referees and editor that this negative result is worth publishing” One can see this as a special case of: you will need to do a good job convincing your readers that this negative result is worth reading.
    – PLL
    Sep 20, 2018 at 18:59
  • 7
    @PLL Your readers is a vague and recursive definition leading to logical fallacies: if they are not convinced, they are not your readers, so all your readers are convinced, etc. Reviewers and editor is well-defined set.
    – Dmitry Savostyanov
    Sep 21, 2018 at 8:35
  • 3
    Another famous example is Minsky and Papert describing limitations of the Perceptron.
    – BartoszKP
    Sep 22, 2018 at 17:24
  • 3
    I once wanted to apply some technique to a problem in bioinformatics. Some colleagues told me "That won't work", but I couldn't find any publication about this. So I tried it and it didn't work. We ran many experiments using different scenarios and got it published. And I'm happy about it, because now the next PhD student can find my publication and have evidence that this popular method does not work for this problem. It was the main publication of my thesis and not regarded negatively at all.
    – Sentry
    Sep 24, 2018 at 12:25
21

One case I can imagine for that, looks something like that:

Based on existing research [1,2,3,4,5] we hypothesize seemingly obvious idea that process X should follow path Y. But when we look at this super-obscure papers [6,7] it seems that X only follows Y under very limited circumstances. Hence, generalization that X follows Y is false. Here are more details.

In scientific paper you have to create something new, based on something old. For example, new theory that refutes old theory, or new theory that is based on recent experimental results. In this case you base your theory on existing theories/data, and logic. Then refute it using some other, less known data.

In nutrition circles, for example, there is a wide-held belief that eating too much protein will stress and damage kidneys. That seems about right, since if you put too much pressure on some system (kidneys) it will suffer, correct? Well, the data on that comes mainly from kidney-failure patients, and when you look at data on healthy subjects, there is no harm at all.

Improve this answer
4
  • 17
    There is a nice example of this in a field that is closer to the OP's, namely Software Engineering: The Waterfall Software Engineering Process. Here, the author described a process that cannot possibly work, then explains why it cannot possibly work, and lastly proposes modifications to the process to make it work. The important aspect is, that the author didn't pull Waterfall out of thin air, rather he designed it explicitly based on widely-believed falsehoods that managers without a Software Engineering background have about how Software Engineering works.
    – Jörg W Mittag
    Sep 20, 2018 at 18:17
  • 14
    In other words: Waterfall is a process that organizations that are not experienced in Software Engineering conceivably might come up with themselves and believe it to a good idea. In fact, the paper was too successful in this: managers who didn't understand Software Engineering actually did read only the first third of the paper, and implement Waterfall in their organizations, thinking it looked like a good idea.
    – Jörg W Mittag
    Sep 20, 2018 at 18:19
  • 4
    @JörgWMittag I believe it was "1976 Bell & Thayer - Software Requirements are they really a problem?" that popularised the incorrect reading of Royce's original paper
    – mcottle
    Sep 21, 2018 at 5:59
  • 2
    Another example is Grothendiek's "Hodge's General Conjecture is False for Trivial Reasons" (pdf).
    – Mark
    Sep 22, 2018 at 1:11
10

Other answers don't seem to mention that individual journal policies are important too, and you need to pick the right journal that will publish this kind of research. Obviously you're not going to get into the top journal of your field with a negative result, however, 'PLOS ONE’s broad scope provides a platform to publish primary research, including interdisciplinary and replication studies as well as negative results' (source: https://journals.plos.org/plosone/s/journal-information).

Publishing can take a long time. The main practical question is that given the low impact of publishing negative results, is it worth your time going through all the hoops of getting it published.

Improve this answer
8

Research that contains negative results about an experimental technique that seemingly fails to produce any useful result, or measure what it was trying to measure, can sometimes lead to a scientific revolution.

So yes, if the research addresses interesting questions and is carried out in a competent way, of course it can be published.

Improve this answer
7
  • 7
    This is hardly a fair example. Michelson-Morley was not testing an idea that M and M "had just thought up one day." They (and other experimenters) were testing one of the hottest contemporary topics in physics, in order to distinguish between several competing theories. (And their first experimental attempt was partially inconclusive in meeting that objective!)
    – alephzero
    Sep 20, 2018 at 15:49
  • 21
    But there is a huge difference between disproving a widely-held belief (e.g., Michelson-Morley) and disproving an entirely new idea that nobody knew about until somebody wrote a paper both proposing it and showing that it doesn't work, which is what the question is about.
    – David Richerby
    Sep 20, 2018 at 17:15
  • 2
    @David I don’t deny that there is a difference, but I think that there’s also a similarity. The point is that negative results can have value. Even coming up with a brand new hypothesis and then disproving it can be interesting and worthwhile, and hence publishable. Also, Michelson-Morley didn’t “disprove a widely held belief”. Rather, their result simply didn’t make sense within the paradigm of the physics of the time. They didn’t claim to have disproved anything, but simply reported their findings. Anyway, my answer may not be the most relevant, but I think there’s a useful lesson there.
    – Dan Romik
    Sep 20, 2018 at 20:26
  • 3
    @DanRomik OK but the important thing is that Michelson and Morley were working on something that everybody already cared about.
    – David Richerby
    Sep 20, 2018 at 23:42
  • 2
    David is completely right. M&M didn't come up with an idea that didn't work ("we propose aether theory and we show experimentally that it's inviable"), they proposed a separate idea ("you can use interferometry to measure the aether wind") which did work.
    – E.P.
    Sep 21, 2018 at 16:09
4

You invented a new hammer and discovered that it is not good for driving in the intended nail. Are their some nails that it is good for? If so, you could write a paper about that -- and bring in the failure to work with the other problem as a limitation of the new tool. On the other hand, if the new tool is literally good for nothing, why bother?

Improve this answer
2
  • 3
    well, because if it is an obvious hammer that saves other people the hassle of finding out that it doesn't work. The more so as not being in the literature, that hammer will stay novel (i.e. may lure sientists because there seem to be possibilities of publishing results obtained by that hammer - before they realize it doesn't work).
    – cbeleites unhappy with SX
    Sep 21, 2018 at 17:53
  • 2
    @cbeleites Good point, but if it is an obvious hammer then it is probably a hammer which comes from applying some hammer-making heuristic, in which case the limitation of that heuristic could be the focus of the paper.
    – John Coleman
    Sep 21, 2018 at 17:57
4

If the idea is interesting enough, and the reason why it doesn't work is also nontrivial and interesting, then I would say this is definitely publishable, with bonus points if you can use the combination of idea-plus-no-go-theorem to shed some new light on aspects of the problem which have not been unearthed before.

Basically, I would propose as the key criteria to frame this decision:

  • How likely is it that someone else will come up with that idea in the near- or mid-term future? How likely is it that a literature search will save them a significant chunk of time?
  • To what extent do the techniques you use to prove your no-go theorem add new tools or insights into the theory?

If you think that the answer to either of those questions is positive, and you feel confident that you can convince both editors of reviewers of that answer, then I would say that you should go ahead and try it. (Of course, this involves a huge judgement call, and reasonable people can disagree about both of those answers. If you're in doubt, consult with colleagues before going ahead.)

In fact, one of my papers (this one) arguably falls into that category: there was this existing observation from experiments ("there is an asymmetry in the harmonic emission doublets") and we came up with what we thought would be an excellent way to explain it ("if you analyze from a rotating frame, there are ionization-potential shifts which simplify the ionization dynamics"), but it turned out that in the full analysis the simplification brought in by this idea is exactly cancelled out by some non-intuitive effects.

Of course, that's not the whole story, and there are a bunch of conceptual insights into the configuration and the overall theory that fall out of the analysis, as well as some concrete experimental predictions, so you can't cleanly say that the idea-plus-disproof by itself is what made the paper publishable. But then, I would argue that if the work is nontrivial enough, then it will rarely be the case that you can cleanly separate the idea-plus-disproof from the conceptual insights added by the paper, and it is the latter that can really make it publishable.

Improve this answer
4

Dmitry has covered it well, so here is an example of showing a mainstream direction does not work from the field of computer science, and specifically time series data mining. The paper Clustering of time-series subsequences is meaningless: implications for previous and future work (2004) by Eamonn Keogh and Jessica Lin at UC Riverside showed that online and offline clustering of time-series subsequences via a sliding window was completely meaningless and led to incorrect conclusions.

This is a good paper to reference since it motivates the problem with a review of the literature and that many of the papers' conclusions would be invalidated by this negative result, performs a number of experiments to demonstrate the negative result, provides a remedy to the negative result, and concludes.

Improve this answer
3

You have to realize that the publishers motivations are not purely academic. I’ve tried to publish null results before (on a topic of high interest) with negative outcome. There are two problems you face.

First, if you publish on a completely new idea (that you prove ill founded), there just isn’t a wide audience. If you seek to prove Relativity wrong, you’ll get attention. But, if this is totally new, the readership won’t care much.

Second, in today’s journals, you need to tell a nice story. The reaction to my own attempts were basically ‘we believe you, but come back when you can end on a positive note’

So proving x != z when no one is studying x or z isn’t interesting, but even if it were, it’s more compelling to outline that x != z, but x ~ y ~ z.

Improve this answer
2

I would like to add to previous answers, that if your idea, if realised, is useful, then if you stress what exactly stops your idea from working and what issues (potentially olvable) need to be solved for it to work, then it is a valid and useful paper.

Example: Orbital elevator. Possible? Not currently. Interesting? Yes. Scientific value? Depends on how deep the research is done.

Improve this answer
2

Think of publishing papers as a way of saying "Hey, here is something I've proven, rather than do this yourself you can read my results and see if they help you out in what you're thinking of doing." to help everyone progress.

Now, the question is, is your 'never-existing idea' something other people might try? If not then you aren't telling them anything useful and it isn't worth publishing. Otherwise it is worth it.

Improve this answer
0

There are Journals of Negative Results, e.g.

http://www.negative-results.org/

https://link.springer.com/journal/12952

http://www.jnr-eeb.org/index.php/jnr

https://openaccesspub.org/journal/ijnr

Thus, generally, publishing "negative" results is possible. Choose a journal appropriate for your field of work and check its "Guidelines for Authors".

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .