28

What can be done to correct an error that might be hiding a certain author's sole purpose of publishing a paper, no matter the soundness of their method?

In a nutshell, the particular case that spawned this question can be summarized in a few sentences:

  • a paper presents a mathematical construction and proposes a numerical solution for the resulting nonlinear optimization problem
  • the problem has many local solutions that are not numerically distinguishable
  • it is impossible to ensure convergence to the real solution without actually starting near it
  • there are too many local solutions (an infinite set in some cases) to simply test the best sample
  • the authors claim unconditional convergence to the true solution, regardless the parameters
  • their results are consistent with their goal, but the mathematical formulation is in contradiction with the conclusions simply judging by the properties of the objective equations. Hence, such results are, at best, a stroke of luck, if not part of a probably much different approach.

After spending almost one month trying to understand not one, but a series of three inter-linked articles on a scientific subject, I have mathematically proven that the central assumptions and claims in that series of papers were wrong and/or incomplete. As the papers were peer reviewed and even published in reputed journals or conferences, I feel there is a need of saving other people the trouble of wasting valuable time trying to reproduce a falsely advertised behaviour. What is the proper action to take in this situation (for example, writing to the editors of the journal)?

Improve this question
5
  • 9
    You might be right but you might be wrong as well. If we are talking about 3 articles, one or more authors and at least six reviewers already disagree with you. Are you 100% sure the result is wrong? Can you be 100% sure for that? Are there previous works that you can base your proof on? Because " I have mathematically proven that the central assumptions and claims in that series of papers were wrong and/or incomplete" sounds like a contradiction, since wrong <> incomplete.
    – Alexandros
    May 7, 2015 at 17:05
  • 4
    @Alexandros to be fair, reviewers don't check correctness. (Maybe in some fields, but not all.) In my experience, in high-energy physics, reviewers check that the paper is internally consistent and that the authors demonstrate awareness of the current state of the field, and they also evaluate the paper on its originality, relevance, and suitability for the journal. They may also pick out some obvious errors. But, assuming that the main result is either plausible or well-justified, they tend to take the authors' word on its correctness.
    – David Z
    May 8, 2015 at 7:45
  • @Alexandros I agree with David Z on the matter, myself having spotted a mistake in a reviewed paper of mine and corrected it in a revision. Reviewers do not have the time to redo the mathematics (unless they're really skilled in that field and care for that particular problem or if it's obvious enough). While redoing the derivations myself, I corrected a few minor mistakes, probably type-os. These are very common and usually do not change the outcome unless they were used in the authors' implementation. A reader is almost always responsible for redoing the maths.
    – teodron
    May 8, 2015 at 7:50
  • To sum up: the problem was discussed with 3 of my research colleagues already and we found the same problem, but couldn't fix it ourselves as it's obviously from a category where convergence to a solution is never guaranteed. The proof to this fact can be found in nonlinear optimization textbooks, once the problem is proven to be part of a particular family of such functions (which it is). It's more a matter of whether discussing this error in a paper might be interesting enough for the community, while a short proof could simply be posted on arXiV in no time, but have less "impact".
    – teodron
    May 8, 2015 at 7:53
  • 1
    Even after the update, it's not at all clear to me what "an error that might be hiding a certain author's sole purpose of publishing" means. And, although I don't know what it means, it sounds a lot like you're alleging deliberate misconduct. If that is what you are alleging, you need a heck of a lot more evidence than "the paper contains errors".
    – David Richerby
    May 8, 2015 at 21:33

5 Answers 5

Reset to default
35

Write a paper explaining what the errors are and how they invalidate the results of the papers in question, then submit it to a journal with good visibility and get it published. Writing directly to the journal editors is appropriate only if you have good evidence that the errors in question are deliberate (e.g., the authors have fabricated data in order to obtain the results they wanted).

Improve this answer
10
  • 24
    Addendum: you need to keep in mind that a large majority of the errors you encounter in primary research are not deliberate. Authors and referees are not infallible, and sometimes errors simply get through the system without nobody noticing until after they are published.
    – Koldito
    May 7, 2015 at 14:59
  • 24
    Accusations of fabrication are a very serious matter, and if they turn out to be unfounded, your reputation will suffer a lot. Unless you have hard evidence that fabrication really did happen (i.e., someone from that group comes out to you and confesses that it did happen), it is better to treat the errors as unintentional errors. It wouldn't be the first time that astounding howlers get past the reviewers.
    – Koldito
    May 7, 2015 at 15:04
  • 4
    Alright then, since accusing the authors of misconduct is not the focus, then I'll go for trying to publish the proof that their method is incorrect, leaving other authorities to decide what is best to do. Thanks a lot for the pointers!
    – teodron
    May 7, 2015 at 15:07
  • 25
    @teodron Math is hard. Math errors are easy. Don't attribute malice where mistake can explain the observed behavior.
    – jakebeal
    May 7, 2015 at 15:34
  • 14
    @teodron It's more improbable that a minor error could have caused the right results You'd be surprised, especially because there's often a lot of back-and-forth between analysis and simulation. Having a theoretical mistake and an empirical bug come into coherence most certainly happens, even though it's much rarer than either happening alone.
    – jakebeal
    May 7, 2015 at 15:50
29

I have mathematically proven that the central assumptions and claims in that series of papers were wrong and/or incomplete.

This is a very complicated statement and it is important to understand it in order to know what to do. The two issues are assumptions and claim.

People often make assumptions to solve difficult research problems under the assumed conditions. If your assumptions are too extreme, or worse known to be wrong, then no one will care about your solution. The key issue, however, is that if someone later proves that your assumptions were wrong this does not make your results wrong. It just means that the conditions for which you solved the problem are uninteresting.

Given a set of assumptions, regardless of if they are true or untrue, claims based on those assumptions can be correct, incorrect or incomplete. If in a further investigation you realize that the claims are incorrect or incomplete, or the assumptions needed to obtain the claims are either incomplete (i.e., you need additional assumptions) or incorrect (i.e., the wrong assumption was made not an assumption that was wrong). In these cases there is an issue with the research that should be corrected. Most journals have mechanisms for correcting errors or at least alerting readers to errors.

The concept of proving an assumption to be wrong is a strange idea. An assumption is an idea that you take to be true while an idea that will be subjected to testing is generally called a hypothesis (or in mathematics I believe a conjecture). Taking someone else's assumption and hypothesizing that it is true (or false) and then testing that can be very valuable research. Proving that the assumed conditions do not occur reduces the importance of the previous research that assumes the conditions occur, but it does not change whether that previous research is correct or incorrect. In this case you need to decide if the proof that the conditions do not occur is interesting enough to publish.

Improve this answer
2
  • 1
    I admit to have blurred the lines between hypotheses and assumptions. In all, the problem is like a recipe that is conceptually flawed, but still attains the goal. As you suggested, it's probably best to carefully write down the scientific proof of why certain claims are in contradiction with the used methods. It will all boil down to whether proving that a particular problem harbours a well-known case where there's no solution and there are insufficient bits of information to workaround that drawback. My hope is that anyone else who might be tempted to use it in practice will not attempt it.
    – teodron
    May 8, 2015 at 7:42
  • Also, if their paper proves that P → Q, and you prove ¬P, then you've just proved their paper correct, not wrong! False implies anything.
    – wchargin
    May 8, 2015 at 18:20
7

Basically there's nothing you can do. I had exactly the same experience with a paper written by some Ivy League computer scientists whose algrotihm I was supposed to implement. Their papers contained no information about how they chose the starting points for their optimization, which was a serious problem because there were many local maxima. They had written a software package but it had been withdrawn from circulation.

I discussed my problems with other researchers in the field and was told that it's generally known that "there are problems with that paper." That's as far as it goes, really. It would be nice if there was some way of calling them to account for wasting peoples' time and making claims that they couldn't substantiate, but there's really nothing you can do.

Improve this answer
1

Of course, you could write a paper proving the original work wrong and try to get it published. But experience shows that such papers tend to have a hard time in review, because they challenge a (more or less) widely accepted method. Just consider, even if the method is not guaranteed to yield the optimal result, it may still be "good enough" to be useful for many purposes. Simply refuting the method is thus only of limited use for the scientific community - in absence of a better method they will just keep on using it. Your paper may end up being simply ignored.

Ideally, you'd start from your proof that the method doesn't work and develop an improved method that avoids some of the shortcomings of the original method. If you can show that your new method really improves performance, you'll be in a good position to publish with good visibility. Such an approach would also be a lot more useful for the scientific community, because you don't simply refute the original, but improve on it.

And make sure you publish that code under an open source license in a persistent, public repository, so everyone can use and improve it.

Improve this answer
0

Approach the authors to explain them the issues and ask them how to work together on publishing a rebuke of their own papers. If possible, do so with the help of someone you know who knows them.

Unless you are an authority in the field, in which case you can do whatever you want, it's much easier and more productive to "withdraw" published claims with the authors' collaboration. Also, you might earn a long-term collaboration with their research group.

Suggestion based on how a professor I know (having a h-index around 50) solved such an issue.

Improve this answer
4
  • In case you didn't notice, there is a series of three inter-linked articles on a scientific subject, in this question.
    – Nobody
    May 8, 2015 at 5:20
  • @scaaahu: so? Are you assuming malice?
    – Nemo
    May 8, 2015 at 6:25
  • No, I wasn't assuming anything. The purpose of my comment was to point out that there are three papers. The complexity of the solution you proposed is quite a bit. BTW, I was not the one who downvoted this answer.
    – Nobody
    May 8, 2015 at 6:54
  • @scaaahu ok, I had not understood your point. IMHO that's all the more a reason to try a collaborative approach: three papers (possibly on different publications) may require three separate fronts of work (e.g. with the respective reviewers or journals), while the asker says they are from "a certain author", so working with said author can address all at once.
    – Nemo
    May 8, 2015 at 6:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .