What should I do if actually I found a serious flaw in someone's PhD thesis and an article derived from that PhD thesis?

Question

Recently, I came across a PhD thesis and an article derived from that PhD thesis and found a serious flaw in them that actually makes their conclusions invalid.

In the first place, I was unsure if I'm right or not, so I contacted the guy that wrote the PhD thesis and subsequent article and described my concerns and I showed my calculations to him and asked if he thinks in fact there is a major problem in his PhD thesis and that article, which is derived from it. He responded back that embarrassingly, in fact I'm right and there is a problem but he can't do anything about it.

I didn't want to force him to do something but I'm just thinking maybe there might be a possibility to publish an erratum for at least that article. It's a major flaw and unfortunately it just invalidates the major conclusions of paper and PhD thesis. Surprisingly, the article is cited for 15 times and nobody found that obvious problem. My question: What's the best way to deal with these kind of situations that even author himself/herself admits that in fact there is a major flaw in his/her thesis or article?

Answer 1

One possibility is for you to write a paper on your reanalysis. Often this would need to be something more than pointing out the flaw alone, but that may be discipline-dependent. If the flaw you have detected is a key part in your own analysis of your work, including your updated analysis in a larger article (while citing their paper) is a smooth way to point out how you had to change the method.

The second possibility is that many journals (at least in my area) allow for Comments on articles they have published. These are for just this purpose - to point out something 'wrong' about a paper that has been published. If the editors decide to proceed with it, the original authors are usually given an opportunity to craft a Reply piece to agree, disagree, alter, whatnot. Then they are published back-to-back in the journal.

Answer 2

Do not overthink it.

I have had similar experience and authors were reluctant at first (mostly because they have moved to another problem).

It is very dangerous to leave a flaw in the literature. Especially a critical flaw where others may build on it. This will lead to more chaos. I am aware some (well-known) people do not care about their old results being incorrect. But this is not what Academia is about. You should appreciate the peer review and rigorous findings more than anything else. You already have contacted the author, offer him a collaboration (if seems right) on correcting the whole thing in the context of your work.

You usually have spotted the error because your work is related. On that particular connection re-analyze the claim and correct it. Include in your new paper the claim, prove why it is incorrect and prove the new correct result. This actually can be seen as a new contribution to your paper/thesis.

For me, I was very satisfied with the outcome; the first author of the other paper did mentor me for a while after we met at a conference.

Answer 3

Write to the editor of the journal. Offer your explanation and ask for the best way forward. You might be lucky and be given a chance to write an article with your clarification. The editor might instead decide to do a correction with varying levels of acknowledgement of you.

article is cited for 15 times and nobody found that obvious problem

The problem is not obvious despite your initial impression. It may be worthwhile dissecting and contemplating the complexity of the blindspot in this area. Was it obvious to you because you come from a different perspective? Have experience from a different discipline? Interrogate why the issue was not obvious to at least 15 publication worthy authors and not to mention a larger number of editors and peer reviewers.

Answer 4

This is what science is. Not everything that gets published is correct or true. Presumably, the literature of science is self correcting. Someone will eventually address and correct it. For example, Descartes' whole physiological framework is just about entirely incorrect. The process of science fixed this, over hundreds of years, and still Descartes publications were an important part of that process.

If the area of research you're referencing is important, the error you hold to be present will misguide the community and harm future research efforts, and you're a participant in the community with sufficient credibility and standing to correct the issue, you might consider a letter to the editor or a rapid publication to a high profile journal.

If any of those criteria do not hold, your actions should be tempered. If the publication you believe to be incorrect is on the topic of your personal research, the issue might find its way into the discussion section of one of your near-future publications. If it doesn't reach that level in your personal research portfolio, you might just let it go, and let someone more appropriate address it.

Answer 5

I have been in a similar situation, but on the other side. My own MS thesis and an article derived from it contained a major flaw. In short, the thesis was about investigating some static waves in a certain system, but the system itself is unstable in the very same model, with the instability length being comparable to the characteristic wavelength of the static waves in question. The whole investigation did not make any scientific sense at all, because the assumed system could not be physically realized in the first place.

I myself discovered the flaw a few months before submitting my thesis. I discovered the flaw pretty accidentally: I wanted to formally prove that the system is stable, but the result of my calculations showed that the opposite is true. I had never heard from my supervisor, who had given me the problem for my thesis, or from his colleagues that the system may be unstable. Everyone simply did not even think about that possibility.

And then I faced the dilemma as to what to do.

My final choice was to tell no one and to simply get my degree. I was an undergraduate student graduating from a Russian university, and getting my MS degree asap and moving abroad to get a Western PhD degree was my highest priority. If I had raised the issue about the flaw, I would have suffered a major setback in my career. I would have had to start my MS project over, and I am not even sure whether I would have been allowed to do it at all. If I had had a setback in my career, I would not have been able to win a prestigious PhD stipend in a Western country.

I should have checked at the very beginning of my MS project whether the system is stable. I should have, but I did not. Partially it is a fault of my supervisor, who directed my work in a very rigid way, giving me concrete tasks and deadlines. He never told me to check whether the system is stable. The official research plan, which I and he signed, did not contain any mention of a stability analysis. It was my own initiative to try to prove stability, because I felt that this was needed to make my investigation complete. I did not even talk to my supervisor about my idea to perform a stability analysis. After I discovered the flaw, I was sure that if I talked to my supervisor about the flaw, he would say the whole MS project had to be canceled.

After I submitted my thesis, my supervisor insisted that I write and publish an article derived from the thesis. I did not want to do it, but I had to. After all, I needed good recommendation letters from my supervisor, so I had to obey. The article was published in a reputable American journal and was cited ~20 times. Writing that article was the most unpleasant experience in my scientific career.

No one noticed the flaw, so I successfully got my MS degree in Russia, moved abroad, got a Western PhD degree, and some years later published an article explaining the flaw. I explicitly wrote in that article that my previous article and a number of other articles, which I cited, are invalid science. No one published a comment in response. In private conversations, my colleagues confirmed that my conclusion about the flaw is correct.

Why did I wait about six years to tell the scientific community about the flaw? Don't hate the player, hate the game. I simply could not afford a career setback. I needed my MS degree asap. I believe I made a wise choice. I would do exactly the same in the same circumstances. Who wouldn't?

Concerning your question, the principal thing is this: what's in it for you? Does the guy's article make any harm to you or your reputation by, e.g., contradicting your own articles? If it does not, then I want to tell this: there are so many wrong or misleading articles in science, there are so many awful things in science, and the guy's article is a drop in the ocean of all this. Why wouldn't you focus on doing your own great research instead? I guess the guy's article is insignificant anyway and is not even worth considering, just like most articles in science. I guess it is just a mathematical or computational exercise made for the purpose of producing a paper and earning a PhD degree. If my guess is correct, then you do not need to stoop to that level and search for mistakes in insignificant articles, and you do not need to harm the guy's career at its very beginning. I have articles published in Physical Review Letters, even as the first author, but I started my career with a wrong and misleading MS thesis.

Imagine you noticed a mistake in an article published in Indian Journal of Physics, whose impact factor is below 1. Would you care to correct the mistake by publishing a comment? I doubt you would. Not every mistake must be corrected. Some are just not worth spending time and effort correcting them.

My own principles are simple:

1. All physics journals that are "lower" than Physical Review journals are trash bins.

2. If an article does not interfere with my own research, then this article is none of my business.

I hope that my post will help you and others look at the issue from an angle different from the one from which many people see the issue.

Answer 6

Surprisingly, the article is cited for 15 times and nobody found that obvious problem.

Not so surprising, my friend, as many people cite papers and books they have not read well, or read at all.

You can email the author, outlining the problem. They may or may not be receptive to your argument, they may or may not be hostile in their reply (or reply at all), and they may or may not be moved to publish an erratum. But you will have done all that you could.