27

I have observed that in rebuttals (conference and journals) the author-side is usually very polite, positive, and submissive. Some phrases I commonly read are

  • "Thank you for the detailed improvement suggestions. We will incorporate them into our work."
  • "We are happy to read that our work was well-received."
  • "We will tone down that claim as you suggested"

Some less common phrases are

  • "We acknowledge your concerns about ..."
  • "We believe you may have misunderstood ..."

According to my understanding, authors usually choose this polite and submissive tone because they fear upsetting reviewers, who are usually in a more powerful position during a rebuttal than the authors.

Question: Have you ever observed a case where the authors were successful with a direct, offensive, and combative strategy in the rebuttal? I rarely every read rebuttals like

  • "Your evaluation of our work lacks objectivity and is scientifically unsound."
  • "Your assessment of our work's novelty is not supported by sufficient evidence."
  • "It is clear that you are not as familiar with the subject as indicated by your reviewer confidence score."
  • "The papers you suggest us to cite are not related to our work, which raises the suspicion that you are trying to unethically promote your own papers."

Has anyone ever tried or observed this strategy and was it successful? If yes, how did the authors do it? How did the reviewers react? I am wondering about this because I always find it a bit awkward to be very polite and submissive. Please note that I am aware of the typical recommendations of how to behave during a rebuttal (e.g., as discussed in this question).


Please note that I am not asking whether any of these phrases are morally acceptable or not, or whether you think this is a good idea.

1
  • 1
    Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on Academia Meta, or in Academia Chat. Comments continuing discussion may be removed.
    – cag51
    Apr 20 at 17:33

10 Answers 10

5

I enjoy reading reviews and rebuttals in journals that practice open peer review. I have seen responses that are very...blunt, so for the exact question ("does anyone do this ever"), yes, that's a thing someone has done ever. One that sticks in my memory is a very short negative review (the words "garbage in, garbage out" were used multiple times) where the author replied that they felt the reviewer hadn't taken it "seriously", and were going to exclusively respond to the comments of the person "who bothered to actually review it" (and wrote in far more detail). This review was a couple bullet points with few actionable comments, while the other review had far more to say.

As alluded from the fact this stuck in my memory, it's an outlier. I've never seen something as far as the examples in the OP, and agree with the assessments in other answers of their appropriateness/lack thereof. With regards to the original question ("has there ever been a circumstance where people criticise the review openly"), yes, but not to the extent of e.g. accusing colleagues of possible misconduct or taking shots against their presumed identity. Even the extreme comments like this were technically "comments on content rather than contributors".

70

According to my understanding, authors usually choose this polite and submissive tone because they fear upsetting reviewers, who are usually in a more powerful position during a rebuttal than the authors.

Perhaps some do, but the reason why I choose a polite (not submissive) tone is that I appreciate that, at least the vast majority of the time, the referee has volunteered in good faith a significant chunk of their time to read my manuscript. My manuscript is not God's gift to mankind, no-one has any obligation to read it, so I am grateful to people who do volunteer to read it and assess whether it can be published. If I think they are fundamentally wrong, I will say so and explain why. But I have only very rarely had any reason to do so. Much more common is the situation where the referee's misunderstanding points to aspects of the paper which ought to be improved.

Your proposed rebuttals add nothing of value to your response to the referee. Clearly it's not sufficient to just say:

Your evaluation of our work lacks objectivity and is scientifically unsound.

You also need to explain precisely why it is scientifically unsound. But once you have done so, there is no reason to add pointless biting remarks like this: your factual rebuttal is what will demonstrate that the referee's evaluation lacks objectivity, not your statement that "your evaluation of our work lacks objectivity", which has zero value to either the editor or the referee and makes objective discussion more complicated.

You are conflating being assertive in the sense of clearly explaining why the referee is wrong and attributing negative motivations or incompetence to the referee. Note that people can be wrong, and frequently are, without being incompetent. Moreover, if you find yourself attributing incompetence to referees most of the time (as I infer from your previous question), then chances are that what's really happening is that you are having trouble seeing your own manuscript through the eyes of other people (rather than most of your referees being incompetent). This is a problem that everyone faces to some extent, but getting into the habit of putting the blame on the referee will not help you get better at doing this.

Finally, there seems to be a clear connection between the attitudes you exhibit in this question and your attitude of "hating your peers" which you exhibit in this question. You seem to be overeager to attribute negative qualities to your colleagues: not only incompetence to referees (which may be sometimes but not typically warranted), but also "submissivity" and "fear" to people who respond politely to referees. Overall, I predict that your career in academia will not be a very satisfying one unless you manage to get over this pronounced need to attribute negative motivations to others.

6
  • 13
    +1, generally good answer. However I don't think personal remarks and what the thread opener has written elsewhere belong in an answer, which is also for others who may have the same (legitimate) question. I think this bit should rather be a comment. Apr 19 at 12:02
  • 25
    @ChristianHennig I understand that this is not how Academia SE is supposed to work, but in case the underlying issue differs from the literal question, I prefer to write answers addressing what I find to be the underlying issue for the concrete person asking the question. In such cases I feel that writing primarily for the benefit of the concrete person asking the question is more useful than writing answers primarily for the benefit of an unknown abstract audience. Apr 19 at 19:21
  • Please note that I asked a different question. Have you seen aggressive rebuttal strategies be successful or not? As stated, "I am not asking whether any of these phrases are morally acceptable or not, or whether you think this is a good idea".
    – mto_19
    Apr 22 at 14:14
  • 1
    @mto_19 I think it would be very irresponsible to not warn someone of the consequences of responding like this. The answer is trying to help you. You don't have to listen but the consequences will be yours, not theirs.
    – Bryan Krause
    Apr 22 at 15:41
  • 3
    @mto_19 "Please note that I asked a different question." Your question seems based on a misunderstanding, so I addressed the misunderstanding. "I also really don't like that you try to take this ad-hominem via my other questions." Yes, it's almost as if introducing a personal angle has the potential to irritate people and distract from the matter at hand. I wonder if this phenomenon perhaps bears some connection to your question... Apr 23 at 6:41
55

I have been successful in forcefully rebutting claims by reviewers on several occasions for both papers and grants. I've convinced editors/grant panels to ignore a review, and to commission further reviews. But it's generally not by saying things like:

  • "Your evaluation of our work lacks objectivity and is scientifically unsound."

  • "Your assessment of our work's novelty is not supported by sufficient evidence."

  • "It is clear that you are not as familiar with the subject as indicated by your reviewer confidence score."

  • "The papers you suggest us to cite are not related to our work, which raises the suspicion that you are trying to unethically promote your own papers."

Firstly, these address the reviewer, not the person making the decision (the editor, or panel member). Secondly, they have a whiff of ad hominem about them. That is, they attack the person, not the argument. Thirdly, they are statements without support themselves. Now, you might go on to provide support, but if you have provided sufficiently convincing support, then the summary statement should not be necessary. As writers say "show, don't tell".

Instead of "Your assessment of our work's novelty is not supported by sufficient evidence", say "The reviewer states that our work is not novel, and support their point with reference to the work of A, B and C. However, our work differs the work of these groups in the following ways... Thus we believe that our work is novel".

Instead of "The papers you suggest us to cite are not related to our work, which raises the suspicion that you are trying to unethically promote your own papers.", say "The reviewer suggests we cite A, B and C. However, our work is about X, and A is about Z, B is about W and C is about Y. Thus we do not follow how they are relevant."

Note that the above is neither submissive or aggressive. It's just a list of statements which you hope the arbiter will take as facts. Your job here is to seem like your are more objective, less emotional and have a better command of the literature and intricacies of the techniques than the reviewer, without actually saying so. If the facts support you, saying so is unnecessary after you've laid out the facts. If that doesn't convince the editor/panel, then saying you are better than the reviewer isn't going to convince someone who isn't swayed by the facts.

This is also where the politeness at the top comes in. This helps bolster the idea that you see this as a professional discussion of facts, rather than an angry exchange of opinions. You recognise your "opponent" as a human, who has, after all, given up their time, entirely voluntarily, and without payment, to consider your work. So in that way, they are doing you a favour. It helps to actually believe this. You then go on to show, surgically, in detail and without any defensiveness, using only facts, that they are entirely wrong.

Addendum

I should add that there is occasionally a case for your first comment. And that is when a reviewer makes unwarranted personal comments about you. If comments are made about your gender, race, nationality, age, name, the university you work at etc then you are warranted in replying to such unprofessional comments by pointing out that they are unprofessional. Really, such comments should have been screened out by the editor, and I would write to the editor pointing that the reviewer has attacked you personally on the basis of something irrelevant to the work. I would include a note that you are willing to address any substantive comments the reviewer has (I would address them ahead of this if there are substantive comments).

5
  • 3
    +1 for pointing out to address the review, not the reviewer Apr 22 at 11:10
  • Please note that I asked a different question. Have you seen aggressive rebuttal strategies be successful or not? As stated, "I am not asking whether any of these phrases are morally acceptable or not, or whether you think this is a good idea".
    – mto_19
    Apr 22 at 14:15
  • 1
    @mto_19 I don't comment at any point about the morals, but rather about the effectiveness. I have seen direct or blunt rebuttals, which actively demolish the case of the reviewer be successful. Indeed, I have written them. But I have only seen them be effective when used with utmost care, and give the judge (the editor) the impression you are not being fair and unemotional, while building a picture of the reviewers incorrectness in the editors mind. I advise against ad hominem not because its morally wrong, but because it is ineffective. Apr 22 at 14:48
  • 1
    @mto_19 To put it another way - I have seen, and have been successful with calling an argument in a review "stupid" , but I have never seen anyone be successful with calling a reviewer "stupid". Apr 22 at 14:49
  • 1
    @mto_19 So you asked "Have you ever seen this tactic be successful?" The answer to this depends on what you mean by "This tactic". Above I outline cases where non-submissive or polite answers can be successful. If you mean "have I ever seen direct ad hominem attacks on reviewers work", then, as I set out in my previous comment, the answer is no. "How did the authors do it?" See the majority of my post. "How did the reviewers respond?" If done correctly, the reviewers don't get the opportunity to respond because the editor discounts them. Apr 22 at 15:02
20

The editor is the person who makes the decision, not the reviewers. From my experience as editor I can say that I will not blindly follow the reviewers. It is hard to not reject a paper if all reviewers are negative, but for sure I have seen problematic reviews, and I am open to authors taking a strong position against a reviewer. I have accepted papers where an author was very negative about one review for sure. I may be convinced by the author's arguments; maybe I even had thought for myself already that the reviewer had some unreasonable demands. Not all reviews are of top quality. (As another answer says, however, it's the arguments that count, not making generally negative statements about the review.)

Also personally I don't find the "tone" of the rebuttal very important. Politeness is worth something in life, but as an editor I will not normally be biased against a paper just because the author says straight on that they don't agree with the reviewer's arguments without phrases such as "Thank you very much for this thoughtful review" etc. I will however in most cases send the revision to the same reviewers again and of course I can't guarantee that the reviewer reacts to this in an unbiased manner. But then the fact that a reviewer may be pissed off by the rebuttal and this may come through in their next review will not necessarily make me take their side. At the end I will make up my own mind regarding whose arguments convince me more.

However if I ask for a revision based on reviewers' comments, I also have read the reviewers' comments, and I see some benefit in at least some of them. This means that the author's chances are not very good if they decide to go against all reviewers. Also, in author vs. reviewer controversies I have found the reviewer more convincing than the author more often than not - but certainly not in 100% of cases.

I also add that editors sometimes (or even often) have to handle papers for which they don't have much personal expertise, in which case it is probably more likely that the editor follows the assessment of the reviewers. An editor can spot some reviews of low quality (and bad papers!) even without being expert themselves, but in some cases the editor may not be in the position to know whether author or reviewer is right, in which case there may be a tendency to trust the reviewer, hence reason for the author to try to "appease" the reviewer.

0
19

In answering this question, I want to refer over to my answer on the question "How can I clear myself of emotional attachment when reading reviewer feedback?"

As an author, I am always polite and professional in my response, rather than aggressive, for two key reasons:

  1. Even the most obnoxious and unfair reviewer represents a reasonable sample of the pool of eventual readers for a paper. They're a blustering ignorant fool who doesn't understand the basics of my work? So will many others be! If I want to maximize my impact and persuasion, then knowing which parts upset this ignorant fool reviewer is valuable.

  2. I find that polite and professional responses are much more effective for actually winning my point, if I am actually correct. Saying "Your criticism is scientifically unsound" turns it into an assertion about opinion, which is much less persuasive than a polite query that exposes the unsoundness, e.g.,

    "We are unclear on how the reviewer came to the conclusion that our proof was invalid, as they failed to identify either an erroneous argument or provide a counterexample. We would be happy to address this in further revision if the reviewer can provide a clearer statement of the basis for their conclusion."

There's no submission or strategic appeasement in these tactics, just a recognition that polite and professional communication is the norm for good reasons.

2
  • Please note that I asked a different question. Have you seen aggressive rebuttal strategies be successful or not? As stated, "I am not asking whether any of these phrases are morally acceptable or not, or whether you think this is a good idea".
    – mto_19
    Apr 22 at 14:17
  • 1
    @mto_19 Sorry that I was not more clear - my point #2 is saying that I generally see aggressive rebuttals fail, because they turn a winnable factual argument into "That's just like, your opinion, man."
    – jakebeal
    Apr 22 at 14:54
8

Responses to referees should assist the editor to evaluate the paper

The politeness in responses to referees comes primarily from the recognition that refereeing is a service, so even if the referee has said something you disagree with, or misunderstood some aspect of your paper, they are still doing you a valuable service by reviewing your paper at all. It is appropriate to be polite to other professionals who volunteer their time to assist you in the process of creating a publishable paper for the public. No submissiveness is required, but politeness ought to be a given. It is a basic expectation of professional conduct and basic human decency.

With regard to the examples you give of offensive or combatitive responses, yes these are all offensive and childish (with the possible exception of the first one), but that is not even their major problem. Their major problem is that they raise extraneous assertions that do not give any substantive assistance to the journal editor in evaluating the merits of the paper at issue. This means that aside from being offensive, they do not address the goal of ensuring that the paper is sound, or assisting the editor to evaluate the paper. I cannot imagine any of these statements making an editor more likely to accept a paper, and honestly, some of them might even cause a rejection of an otherwise meritorious paper. Below I give more detail on the problems with each statement.


"Your evaluation of our work lacks objectivity and is scientifically unsound."

This kind of statement (only more polite) might possibly be okay if it were a conclusion stated after making a compelling case for a lack of objectivity in the review, or a fundamental error in scientific reasoning that renders the review flawed. As a stand-alone statement is is merely bald assertion and it does not assist the editor to determine the quality of the paper at issue. Even if such a finding were warranted, there is no reason it could not be couched in more polite language that is respectful to the reviewer and thankful for their efforts --- e.g., "We appreciate the efforts of the reviewer to analyse our work. With great respect to the reviewer, for the reasons previously stated, we feel that he may have lost objectivity in review of the work and may be proceeding on the basis of a flawed view of [subject/topic]. For this reason, we do not propose to make suggested changes X, Y, Z, and we request the editor to accept the paper without these suggested changes".

"Your assessment of our work's novelty is not supported by sufficient evidence."

Why would you expect a reviewer to bear the onus of proving that your work lacks novelty? If novelty is in doubt, the author must explain what the paper adds to the literature that is not already there --- it is not a presumption to be "disproved" by evidence from a reviewer. (Though as pointed out in comments, if the reviewer is aware of other literature that covers the work, they should raise this specifically with relevant citations.) Rather than saying this, if you think your work is novel, why wouldn't you just tell the referee all the reasons that you think your work is novel --- e.g., "We note the referee's concern about lack of novelty. We have reviewed our work against the relevant literature to confirm novelty of the material presented. Our view is that the work adds elements X, Y and Z, which plugs a gap in the literature that is not covered in papers A, B and C."

"It is clear that you are not as familiar with the subject as indicated by your reviewer confidence score."

What possible value does this statement have in explicating the value of the paper at issue to the editor? This is nothing more than an offensive cheap-shot at a reviewer accusing them of being over-rated, with no substantive bearing on the quality of the paper at issue. It carries zero information about the value of the paper or why the paper is written in its present form. It could not possibly assist the editor in any way other than to tell them that the author is childish, nasty and unprofessional.

"The papers you suggest us to cite are not related to our work, which raises the suspicion that you are trying to unethically promote your own papers."

This is a huge accusation to make, and if you're wrong, get ready for a defamation lawsuit. If you submit this, you are accusing the reviewer of unethical behaviour and you are making that accusation through a channel that is read by at least one other influential academic (the journal editor) who is a colleague of theirs. If you think the papers suggested are not worth citing, why not just say that, without speculating about unethical conduct by the reviewer? Academics sometimes recommend citation of their own papers ---without any unethical impulse--- simply because those are the works they are most familiar with on the topic and therefore most likely to know about. (In any case, in a blind review you don't know the identity of the reviewer but the editor does. If the reviewer is promoting their own papers then the editor can see this.)

5
  • 9
    +1 but: "Why would you expect a reviewer to bear the onus of proving that your work lacks novelty?" I do get annoyed by reviews that say something is well known without giving any reference. Unless you are very well embedded in a certain field it is hard to be sure to know all the literature, so if a reviewer says "XXX already exists" I don't think it's too much to ask that they point the author to where this appears. Also reviewers may think something already exists, but in fact there may be a subtle but relevant difference, and without a reference editor and author can't check it. Apr 19 at 13:15
  • I'll slightly disagree, @ChristianHennig . If it's the sort of thing that's widely known enough that I wouldn't provide a reference in a paper, I won't provide a reference in a review either. It may still feel like esoteric knowledge to the author, but that's the risk of stretching outside of your home field.
    – jakebeal
    Apr 19 at 13:49
  • 4
    @jakebeal I'm with Christian here. Lets leave aside whether its esoteric to the author or not, the person who needs to judge is the editor. As a reviewer, you run the risk of it being well known to you, but unknown or esoteric to the editor. Its also the case, as Christian points out, that the work under review might be subtly different in a way that you've missed. ALternatively you just get into a unproductive trading of "Its not novel" "yes it is" between the author and reviewer where the editor has no way of telling who is right. Apr 19 at 14:46
  • 1
    I suspect we're in violent agreement, as I'm thinking about real basics here. For example, last year I reviewed a paper that claimed to solve a well-known problem, but actually had formulated an entirely different and simpler problem. By analogy, it was like claiming to solve the Traveling Salesman Problem, but instead of finding the shortest path to visit each city precisely once, finding any path that visited each city at least once. Given the "101" level of the issue, I did not see any need to provide citations, just a clear explanation of the error.
    – jakebeal
    Apr 19 at 17:21
  • 1
    Just think how many fewer people would've died, if any epidemiologists had ever bothered finding a citation whenever they said "Everyone knows 10 micron particles aren't airborne". Apr 20 at 21:24
8

I've seen hundreds of reviews as an editor, and I don't recall ever seeing one where the authors attack the reviewer's competence in a manner as aggressive as the examples you give. Presumably most authors are aware that insulting the reviewer is counterproductive, especially when they know who you are while the reverse is not true.

However, I've seen some marginal cases:

  • The authors decline to revise the manuscript in light of the reviews. Authors don't strictly have to give a reason when they withdraw their manuscript this way, but sometimes they do.
  • It's also possible the authors write to the editors complaining about the reviewer, either the reviewer's tone or their technical competence. These complaints are not sent directly to the reviewer, for obvious reasons. If the editors agree, the most common thing to do is to disregard the review. This could mean accepting the article even if the reviewer recommends rejection, for example. It also means that, usually, the revision is not shown to the reviewer again.

Writing these aggressive comments in the response-to-reviewer box is asking for trouble. A diligent editor will most likely intervene before the comments reach the reviewer, but not every editor is diligent.

7

Disclaimer: Below I argue for a case in which an "aggressive rebuttal strategy" can be effective. But I don't endorse the idea that any of the example statements from OP's question could be effective, especially if given as stand-alone statements.


In my discipline, there is a major difference between journals and conferences, based on how decisions are made:

  • At journals, the editor makes the decision based on the submitted reviews. While an editor can always overrule a negative reviewer if they don't agree with the criticism, selective journals are typically unlikely to accept a paper that has any negative reviews. So, authors have a clear incentive "to make all reviewers happy", which can lead to submissiveness.

  • At conferences, the decision emerges from a discussion of the program committee members who reviewed the paper. In case of mixed reviews, it can be an effective strategy to emphasize arguments that diminish their credibility of a negative PC member (e.g., pointing out objective mistakes in their review) and provide "ammunition" to the more positive PC members. I have seen various successful applications of this strategy in the past.

2

I will just add that I ended up in a really difficult position with my second paper (of two, so I'm no expert here). I was required to give a pool of suggested reviewers and there was no way I could avoid one particular person in the field, and yet I knew their affiliations were... not ideal, and stacked against my approach.

What came back in his review (I'll never know it was him, but it's 99.9% obviously him) was a lot of ridiculous remarks and it took a 5000 word rebuttal against him. The rebuttal was basically as long as the paper.

I got comments like "The authors claim that this process is done in less than 30 minutes!" (yes, with the exclamation mark) so the response was "The authors kindly point out that no experiment was run for less than 2 hours and, indeed, no graph is shown with results for less than two hours. The authors are unclear where this claim was made.". Not only that, but I frequently used his own papers against his own arguments.

So, not directly aggressive, but a huge amount of basically passive-aggressive, but objective, text to counter and dismantle the spurious remarks.

It worked. I just wish it never happened.

2
  • Most journals will let you list reviewer to be avoided to to a conflict of interest, which is used to avoid exactly this sort of situation. If for some reason the submission site doesn't let you, it's still something that you can put into your cover letter in the case of a severe enough situation.
    – jakebeal
    May 19 at 15:48
  • @jakebeal I cannot remember the absolute specifics because this was over a decade ago, but it was definitely a considered approach. A cursory glance from the editor on Google would have brought this guy's name up as the first result, so we would look disingenuous if we excluded him from the list. We had to bite the bullet.
    – roganjosh
    May 20 at 2:46
0

The phrases you consider can easily fall into the category of ad hominem attacks, where you are criticizing the reviewer as an individual, and not the reviewer's points.

Ad hominem attacks are usually weak, ineffective arguments. Indeed, many consider them to be logical fallacies. Avoid them like the plague. If the reviewer's points are simply wrong, respond that way -- say why they are wrong. If some nonclarity in the document caused a misinterpretation, fix it. Even if your text is clear as day, try to make some sort of nod to the review by attempting to clarify more.

If the whole review seems unfair, then address every single point by patiently explaining why each is erroneous.

An appropriate aggressive response involves addressing the incorrect points with flawless logic. If you want the tone of a response to review to sound more aggressive, leave out all the temporizing language like "Perhaps our description was not clear enough, so we fixed it", and mercilessly shred each of the reviewer's incorrect points with well-reasoned, crystal-clear arguments.

0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .