44

How do you respond to a reviewer who is very critical about your paper but from whose comments you can easily see that he/she has misunderstood the key concepts of the paper?

In my case, the first reviewer had no issues with the paper and suggested only a minor revision. But the second reviewer seems to have not spent too much time on understanding the paper. This reviewer's misunderstanding made the editor suggest a major revision of the paper.

In my response to the reviewer (not sent yet), I used the sentence "This comment completely misrepresent the methodology described in the paper." Is this too much?

UPDATE: I am glad to inform you all that after the second round of reviews, the paper was accepted with minor revisions. I followed the general advice given in the accepted answer for this post. Instead of being too defensive, I agreed in my response to the reviewer that the misunderstanding may be due to wording of certain sentences and changed them. Specifically, I used the following sentence:

"There seems to be a misunderstanding about the methodology, which we hope to clear through our responses below and by revising the text in the manuscript. Contrary to what the reviewer states, ..."

Hope this update helps someone in future.

  • 16
    The mistaken reviewer probably assumed something wrong because the paper doesn't state the opposite. Make that explicit in the paper so that nobody else can do those assumptions after reading that. – Trylks Sep 5 '14 at 9:42
  • 4
    If it is already mentioned in the paper then try to make that mention more clear and then point the mistaken reviewer to that part of the paper. Something like: "We have clarified this further in ___". – Trylks Sep 5 '14 at 11:07
  • 1
    @Trylks: I wouldn't head for that conclusion that the fact was not clear enough in the submission so readily. I too have received reviews of people explicitly stating "The user study has been conducted with 40 users." when the subjects of my study were described in their own subsection in the study description, titled "Participants", whose first sentence read "Our study had a total of ten participants." Among five reviews, there is usually one that contains at least one statement that shows how the reviewer didn't read properly. – O. R. Mapper Sep 5 '14 at 11:21
  • 4
    @O.R.Mapper "I wouldn't head for that conclusion that the fact was not clear enough", the paper may be unambiguous, but "easy to read" is a subjective appreciation, and "clear enough" means very easy to read. For one reviewer it wasn't clear enough, because when he glanced over it he got a false idea. For the editor it's not clear enough, because nothing is enough. You have two options: Either fix nothing and say it's fine right now, or pretend to fix something that was never there and pretend it's much better after that fix, so that everybody can pretend everything is alright. – Trylks Sep 5 '14 at 12:14
  • 3
    Unfortunately, for any level of clarity, there exists a level of reviewer laziness sufficient for a stupid review. Although, to be fair, most of my experience with peer review has been positive. – Sasho Nikolov Sep 5 '14 at 16:41
55

This comment completely misrepresent the methodology described in the paper.

While it may be technically correct, it is unnecessarily undiplomatic (at least all by itself), as it implicitly places blame on the reviewer for not correctly understanding your paper. Again, if the respective points were made unmistakably clear in the paper (which is something that you can easily overestimate), this blame may be justified, but still you risk unnecessarily disgruntling the reviewer.

I would suggest to assume in good faith that the reviewer did not fail to understand your paper due to incompetence or laziness, but because you failed to clarify a certain aspects. Try to improve your paper regarding the explanation of everything that the reviewer misunderstood and reply with something along the lines of:

This comment seems to be based on the assumption that we were proposing a method to transmogrify apples. However, the goal of our method is the transmogrification of bananas – an aspect, which we have failed to make sufficiently clear. We have amended our manuscript accordingly and now write: […]

Do not worry, if the respective revision turns out to be only minor.

To be prepared if the reviewer insists on his or her misunderstandings, I suggest to explicitly state in your response letter that something different is the case. This way you hopefully have some good argument for the editor in this case.

I once had a similar experience with a reviewer who criticised that we made several claims which were not supported by our studies. However, we never made any of these claims. In a first revision we reformulated a few sentences that the reviewer had presumably misunderstood and explicitly stated in the response letter that we did not make those claims. The reviewer criticised again that some of the same claims were unsupported. We responded again that we never made those claims and reformulated a handful of sentences. Then either the reviewer finally understood or the editor was fed up with this and the paper was accepted. In both cases, major revisions were requested.

  • Thank you. That's a very nice way to put it across to the reviewer. – Prometheus Sep 5 '14 at 10:52
  • 6
    Diplomacy is the way to go when responding to reviews... – Floris Sep 6 '14 at 5:29
9

No one ever seems to talk about this but sometimes... reviewers are terrible.

Not terrible like "Ugh they made me revise my paper" but terrible like "Did this person even read the paper?" I once had a paper where a reviewer made lengthy complaints about the terrible user studies and lack of clarity in user studies for an object. There were no user studies in our paper, there were several clear statements about how there were no user studies in the paper, the 'future work' section detailed the user study collaborations. It was a paper on the method of integrating physical and automated controls, not a paper on user studies. One of the reviewers just... lost the plot. I don't know if they read a different paper, I don't know if they wanted a different paper. The other two reviewers made some excellent revision suggestions which we followed.

It wasn't the first time either. I can't speak for all fields or conferences, of course, but sometimes you just get a terrible review. Terrible not negative, terrible as in makes no sense. Sometimes you respond, sometimes you try to work it in and sometimes you just politely ignore the bits that make no sense. Some fields lend themselves to a wide variety of subject matter experts. Consider, for a moment, robotics. A submission to a robotics conference or journal could be about the mechanical engineering aspect, the automation(AI and Machine Learning) aspects, optimization, computer vision, sensor integration... No single person is going to be an expert in all of these things and some of the most interesting, to me, projects involve some blending of these subjects. It can be hard, then, to find a reviewer from the pool who best fits a submission.

This is the case in many subfields in Computer Science, this may be the case in the field you are working in. When you're talking about research on the bleeding edge sometimes you're talking about stuff that has a limited or restricted audience and, thus, a limited or restricted reviewer pool. There are a fair amount of similar questions about academic research reviewers and reviews here. Typically the benefit of the doubt is given that the reviewer is bringing up legitimate complaints and, to be honest, that is the best policy for everyone to have. But sometimes it's ok to accept that a reviewer is totally out there and the review is unhelpful or inappropriate(in that it does not apply.) In those cases I would respond to the other reviews as appropriate and respond, tactfully, to the review in question with statements directly reflecting how the review does not apply to your work.

  • 1
    Yes! I've been surprised by how often questions here get answers that advise being endlessly deferential to reviewers. Sometimes reviewers are incompetent or malicious, and a good-faith attempt to make them happy is exactly the wrong response. – Matt Reece Sep 5 '14 at 19:30
  • How did you respond to the first reviewer you mention? – Antonio Vargas Sep 5 '14 at 21:48
  • 2
    In the specific conference for that you did not respond to reviewers, you just resubmitted the revised paper to the editor and went on your merry way(which seems to be common(ish) in robotics conferences. In another, similar situation where we did have to respond to the reviewers we simply stated that "We have read and understood your review to be a critique of thing which does not appear to be applicable to our submission due to reasons. We have clarified this by blah blah removing any abiguity." – Nahkki Sep 5 '14 at 23:24
  • 1
    @MattReece: If a the only way to respond to the reviewer in good faith is a major negative change to the paper, you may be right. However, in the specific case, the alternatives mainly boil down to 1) “the reviewer misunderstood our paper” and 2) “the reviewer misunderstood our paper and thus we clarified”, the latter of which should only require a minor change. If the reviewer is being malicious, neither of these options will make him happy. Whether the reviewer is malicious or incompetent, 1) gives him the opportunity to cause an additional round by requesting clarification. – Wrzlprmft Sep 6 '14 at 7:32
  • 5
    It's a shame there's no review mechanism for reviewers. The fact that you'll get revenge downvoting is irrelevant as it would be evenly distributed over time, and you could compare to the bell curve. You could have standard questions, too - like "Did the reviewer read the paper carefully? 0=didn't even look at it, 5=wow I wish he was my advisor – Peter Wone Sep 6 '14 at 9:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.