31

A couple of months ago I submitted a paper for which a referee review came back recently, recommending minor corrections. There are two methods to calculate some quantities in the literature which do not agree between each other. My paper agrees with one of them and not with the other, which I know is incorrect. However, in my paper I did not point this out (that it is incorrect, I do mention the other method), but simply focused instead on the method I use.

Now, the referee seems to be eager for me to profusely comment on the incorrect method and specifically refer to several papers written by people from the same group using this particular method. I get the impression that this referee is trying to use me to criticize them. I would prefer to be as diplomatic as possible, and remain in good terms with the others who use the incorrect method, who I have met in person several times. However, openly stating their method to be wrong, even if choosing the wording to be as nice as possible, would make that difficult.

What should I do in this case?

Improve this question
6
  • 10
    However, in my paper I did not point this out — Uh-oh. Don't do that. You don't necessarily need to criticise the other method, but you do need to admit that it exists and differs from yours.
    – JeffE
    Commented Jan 9, 2015 at 17:16
  • 6
    I mentioned it exists, and that it's a different approach. I left it at that, because if I say it's incorrect then I obviously need to argue why.
    – Miguel
    Commented Jan 9, 2015 at 17:20
  • By saying the two methods do not agree, do you mean that they give different outcomes, or only that they arrive at the outcome by different routes?
    – Jessica B
    Commented Jan 9, 2015 at 22:03
  • 7
    Well, since I see from your profile that you're in Physics and doing numerical simulations, I have ask if the formula other guys are using/proposing is provably wrong in the mathematical sense of the notion of proof or just a worse approximation in practice. It kinda hinges on that how you present your case. N.B. I said formula for simplicity, but it's probably something more complex like an algorithm etc.
    – got trolled too much this week
    Commented Jan 10, 2015 at 0:31
  • 1
    @Respawned_Fluff It's a formula that is used outside its range of applicability, based on the approximations it hangs onto.
    – Miguel
    Commented Jan 13, 2015 at 17:54

7 Answers 7

Reset to default
27

In my opinion, having a proof that a method is incorrect and keeping it to yourself due to diplomatic/social concerns is neither the scientific nor the friendly thing to do.

If I was working in a group which uses a method frequently I'd expect a friend knowing that it was flawed to share his/her concerns with me in a constructive and friendly manner, knowing that the next researcher who discovers such flaws might not be friendly about it. I'd also hate to be putting time and effort based on something flawed.

Once that was done by that friend, we could find out between us whether the method is really flawed. If it is, I won't be offended if it's published in a paper he wrote.

Improve this answer
2
  • 5
    If the two calculations give different results, then writing one in a paper gives a proof that the other is incorrect. As I understand it, the OP simply doesn't want to write the corollary 'so the other method is incorrect' explicitly. The reader will have to decide for themselves which they believe either way.
    – Jessica B
    Commented Jan 10, 2015 at 7:41
  • How about talking to them about the different results, instead naming various people using as using an inferior formula in a published paper?
    – Dronz
    Commented Jan 11, 2015 at 8:00
11

First, remember that the review is not "the word of (your) god", it is a professional view of a peer. Now, it is customary to report on aspects that support and do not support your findings. Whether it is reasonable to mention the second, apparently flawed way to do a calculation, cannot be judged here but you should see if it can and indeed should be at least mentioned. In the end, you also have the right to have a different opinion than that given by your peer reviewer. Hence, you should provide your view to the editor on why you think following this particular point is superfluous. It does not appear to change anything in your conclusions and of that is correct it seems as a no-brainer.

So first consider if you can work in a comment to the fact that there are other ways to do a calculation, if need be with an argument why you have chosen as you have done. Second, make your point clear to the editor stating that you do not see a major point in dwelling on the reviewers point since it does not affect the result you have arrived at. How and what you express will of course only be possible for you to judge.

Improve this answer
6

In your paper, you do need to acknowledge the existence of the other method, especially if it is a frequently used alternative. This is because you need to explain why you chose the method that you chose.

You do not, however, need to criticize the other method as wrong. Instead, you can simply explain what are the advantageous properties of the method that you are using. A good way to think about this is to remember that you are not actually choosing between Method A and Method B. Rather, you are choosing Method A as sufficient for your purposes, regardless of the existence of Method B or the opportunity to try to create a new Method C.

Improve this answer
3
  • This is what I did more or less. My question is more about how to deal with the reviewer. The approach I use would be the "mainstream" one.
    – Miguel
    Commented Jan 9, 2015 at 17:23
  • 1
    @Miguel You can still likely say that the approach you choose has certain advantageous properties that the other does now. As for the reviewer... honestly, you just need to be seen to make a good faith effort to take their comments into account. And since it's only a "minor revisions" the reviewer might not even see it again.
    – jakebeal
    Commented Jan 9, 2015 at 17:25
  • 4
    It sounds from the question like the results from the two methods directly contradict each other, and the OP thinks one of them is actually incorrect. In that case, I think the OP should explain their objections to the result they are not using.
    – Nate Eldredge
    Commented Jan 9, 2015 at 17:34
4

We're all in the same pursuit in science - we want to understand what's happening on a fundamental level. If there are disagreements in calculated or measured/observed quantities, I should think that the natural response to reported discrepancies in those quantities should be "Why do we disagree? What is happening here that we obviously don't understand fully?" or similarly. These disagreements can be turned to something constructive and really ought not be seen negatively by anyone worth their salt in academia. Or so I would think.

There was a similar instance in my own work previously. If you're particularly concerned with a personal outcry from these other researchers, what you could do is reach out to them personally to tell them your findings and try to discuss the answers to the questions I stated above. This way, you're free to publish your results and you might even have a "stimulating discussion" that would allow for future work or collaborations on similar measurements. After all - these might lend themselves to better comprehension of your field of study, and isn't that your end goal?

Improve this answer
2

You can contact the editor directly. Explain your position calmly, as you have done here - you feel you have addressed the issue, and that you think the reviewer's approach is too strong, or why it is not a good fit for your paper. In my experience, editors try to be fair, and if the editor will require you to make the change, they will be able to tell you so directly.

Improve this answer
1

Reach out to the group that has published the "wrong" method and discuss it with them. There may be more to their method that you realize. If at the end of that conversation you agree that there is a problem, they ought not to mind if you publish. If you get to see their point of view you can include it in your response to the editor.

Nobody likes to be surprised by a negative publication (so give them fair warning/try to sort it out) and in scientific publishing you should state the truth as you see it, with appropriate language to convey what is fact and what is speculation. Deliberately leaving things out is inappropriate: there is an old saying in Dutch that "gentle doctors make stinking wounds".

Improve this answer
1

Seems like there are three possible cases here.

1) Do you get comparable results with your data, using both methods? Then that's a useful thing to say: "I chose to use this method, but similar results are obtained with the other method". You can leave it at that.

2) If you get DIFFERENT results, then that's rather more important: you need to describe the difference in outcome, and to explain why your cherry-picked method represents a true interpretation of the data, and why you feel that the differing result from the other method should not be heeded.

3) If the method affects data-gathering, so that you can use only one or the other, then you can't compare them without running the experiment again... which is someone else's job! You have already mentioned the other method, and explained which method you have used. Your method is the mainstream one. Papers exist on the topic of which method is better, and about discovered flaws, but your paper is not about this, so you don't even need to cite those papers: it is an irrelevance.

If it is a bugbear for your reviewer, that's nice for them, but it is not relevant to your paper, and you do not need to go into depth about your selection of the most mainstream method. If you were choosing a minority method, an explanation is relevant, but otherwise it's sufficient to say which method you used, so that people can reproduce your experiment.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .