What to do when a referee asks for changes I am not willing to make?

Question

I have submitted a paper later that was reviewed by four reviewers. I submitted my revisions to all reviewers. Three accepted the review, while the last is still asking for modifications.

The problem is that the reviewer is asking to measure the performance of our work using performance metrics that were never taken into consideration before in the literature (and our problem is well studied - at 30 references !) ..

These performance metrics would require a complete Master Thesis.

Is it fair to judge a paper based on performance metrics that were not taken into consideration by the authors at the first hand ? (noting that none of the previous work considered these metrics) ..

I hate to say it, but the reviewer seems to be reading a paper about my problem for the first time ever, and some of the reviewer comments are contradicting.

Note: I am sorry if this have been asked before. I could not find any thing related.

Answer 1

In your question, there is no mention of an editor for the journal to which you have submitted your work. If you disagree with one or more of the reviewers, there is nothing remarkable about that; it is common, a review is an educated opinion about your work, supposedly based on facts. Normally you would provide an account for how you have met the comments (as you seem to have done) and in the cases where you disagree, you provide an account based on facts and reasoning why you think your way is better than that of the reviewer. It should then be up to the editor, not the reviewer, to decide whether your revisions make the manuscript acceptable or not. In some cases the editor will request a second round of reviews, this is normal, particularly if the revisions have been so substantial that the manuscript is quite different from the original.

This is what I see as a relatively typical way for a review process to take place.

Answer 2

I agree with the other comments that it is ok to not satisfy the demands of all reviewers and that in the end the editor makes the call.

However, it is still possible that the reviewer has a good point that the other reviewers missed. It is entirely possible that common metrics in the field are problematic and that since the other reviewers are "from the field" (and so are you, for that matter), they just don't give it a second thought. I have encountered these kinds of situations before. Remember that ultimately, you are the one responsible for what you write in your paper - not the reviewers or the editor. Therefore, I suggest that you keep an open mind and think deeply if the reviewer has a point and how you want to address it and act accordingly.

I am just mentioning this since it is sometimes easy to become overly defensive and dismiss reviewer comments as bad judgement.

Answer 3

Can you elaborate more how the discussion has proceeded? If I am under the correct impression, you don't have to agree with every reviewer nor do everything they suggest. Instead, you need to tell them that their comments have been seriously considered. You should be OK by writing the same statement that you wrote here: you disagree, your point of view is supported by previous research, and so on.

Answer 4

If I may relay a piece of wisdom that was passed down to me many years ago, even if you are convinced that the reviewer is a complete idiot, it only means that your paper was obviously not completely idiot-proof.

If somebody managed to find fault on what you considered a trivial point, then the point may not be as trivial or clear as you think. Applied to your specific case, you could show, in your manuscript, that the results shown are sufficient to make whatever point you want to make, and that other specific tests, i.e. the ones suggested by the reviewer, would not contribute to the result.

In any case, though, my first step would be to talk to the editor in charge of your manuscript. She/He will be making the final decision, so it makes more sense to discuss with her/him directly, and not engage in a lengthy -- and essentially pointless -- battle with an individual referee. In such cases the editor can usually tell you how she/he would like to see the referee's points addressed, and that will give you something to work with.

I also echo Bitwise's sentiment that, in the end, it's up to you. If the journal, via the referees and editor, makes demands that you cannot live with, then you can live without that particular journal.