How to deal with poor scientific articles that seemingly reached the right conclusion?

Question

I am writing a paper and some of our findings validate the conclusions from an already published peer-reviewed article in a respected journal. We also find things that differ.

However the authors of the paper uses techniques that are not ideal for making their conclusions, and furthermore, some of their data look particularly weak to me, i.e. I do not find that they substantiate their claims. For example, there are immunofluorescent images that simply look black.

How do I discuss and reference a paper, that seemingly come to the right conclusions, but do so in a manner, I am not sure about?

Answer 1

I am writing a paper and some of our findings validate the conclusions from an already published peer-reviewed article in a respected journal. We also find things that differ.

That is all good and normal.

However the authors of the paper uses techniques that are not ideal for making their conclusions, and furthermore, some of their data look particularly weak to me, i.e. I do not find that they substantiate their claims. For example, there are immunofluorescent images that simply look black.

Often (especially when it comes to experiments) the choice of technique is down to what is available to the authors, and what their expertise is. This is generally understood, and it's left for later papers to improve on methods, analysis, data, etc. In fact, an earlier less-than-perfect publication, while not great for the scientific record, helps position and justify your own paper.

Also note that it's possible they had better data than made it into the published figures. Sometimes the picture quality deteriorates during the publication process, sometimes the authors do not know how to make high-quality graphics, and sometimes there are outright mistakes. It might be worth reaching out to them asking for more details.

How do I discuss and reference a paper, that seemingly come to the right conclusions, but do so in a manner, I am not sure about?

It's best to be sure before directly criticizing the other work. If you're not sure, it's usually better to focus on presenting your own work in a good light by positioning it relative to the other work. It sounds as if you are in a good position to do that here, perhaps writing something like "System X has previously been studied using technique A [1]. We use technique B [2], which has better time resolution and allows for more precise control of property c." This construction can be adapted to discuss the claims where you agree and disagree with the other work. Basically focus on your own work first and foremost, and argue why we readers should trust it over the other one, not why we should distrust the other paper.

I'll add the caveat that this obviously breaks down if you come to wildly different conclusions than in the existing literature. In that case there's no way around directly addressing their flaws, but then you really want to be sure.

Answer 2

There are at least two aspects to scientific papers, method and conclusion. You can critique either, but should do so using evidence. Likewise you can critique the connection between method and conclusion. There is no real problem with that. But you need to present the evidence to back up any claim so that it doesn't sound like opinion or preference.

On the other hand, if you have come to the same conclusions you may not have to include any discussion at all, other than a citation. It might actually be wise to omit any criticism if the other paper is well accepted. That may not be possible, of course, if you think the validity of your paper in some how depends on the problems in the other. In particular, you need to attack the issue directly if reviewers consider your paper to be just a re-work of known things.

In mathematics there is a huge body of "knowledge" for which all of the published proof is flawed. But we accept that "A new proof of KB's Lemma" as a valid thing without needing to explain, necessarily, how all existing proofs of KB's Lemma are flawed. Sometimes it is easier than others to side-step the issue of course. When reviewers don't see the value of the new work because it is "known" you may need to do more, or, alternately explain to them something about the flaws in the earlier work.