36

In reviewing a paper in analysis (mathematics), I provided a detailed review report that I feel was not too harsh (at least well within the realm of reasonable).

Today, the authors came back with a revision that, in my opinion, failed to address all the major criticisms I raised and ignored most of my suggestions. They included a detailed letter that contains item-by-item response to my points. Most of the responses are along the line of

We acknowledge the point raised by this reviewer, but we decide to not address it at all in this revision.

(I'm paraphrasing, of course. But that's the gist) There is no rebuttal nor any change. As a reviewer, it seems pointless to repeat the same points again. It will just be a waste of time for everyone.

How should a reviewer respond in this situation? Will it be reasonable to decline review invitation?

Note: Of course, I'm not assuming I, the reviewer, am 100% correct in every point. But the correctness is not the main issue here.

Improve this question
7
  • 2
    Devil's advocate: every author knows that their paper is not perfect, but they will act like it is, at submission. Therefore if the reviewers are doing a review to improve the paper, well said reveiwers are wasting thier time :D ... the paper is already in its possible best form! Reviewers are there to find what is wrong in a paper, not what can be better.
    – EarlGrey
    Commented Sep 12, 2022 at 20:03
  • Related: Is it reasonable for authors and editor to ignore review comments without explanation?
    – Snijderfrey
    Commented Sep 12, 2022 at 20:23
  • 2
    Are you saying they ignored suggestions or major criticisms? In math, I would interpret major criticisms as serious flaws in the paper prohibiting publication (errors/gaps in proofs, severely deficient exposition, ...).
    – Kimball
    Commented Sep 12, 2022 at 23:30
  • @Kimball, Yes, gaps in proofs (not necessarily errors) and what I consider rather series deficiency in exploring key ideas. For the journal in question, those should be disqualifying issues. So, certainly not minor in my view.
    – Bilbo
    Commented Sep 14, 2022 at 2:28
  • 1
    @GiuseppeNegro sure, it happened to me as well, but it is fully in the realm of "reviewer working way too much for free". That's why I said "Devil's advocate": reviewers are not supposed to improve a paper, although they do and we are always immensely thankful for some objective commentaries and feedbacks.
    – EarlGrey
    Commented Sep 16, 2022 at 9:28

8 Answers 8

Reset to default
45

Actually, they haven't ignored your remarks, but have decided not to address them. There are reasons both for why this might be reasonable and why not. One possible reason is that they believe that the suggested changes are beyond the scope of this paper as they envision it. They remain the authors.

But if you think the paper is flawed without the changes you can say that in the new report. If you think it is fatally flawed you can recommend rejection. Any of that is appropriate. And, yes, you can decline to review again.

It will be up to the editor to make decisions here. The editor probably has other reports and will need to eventually come to a decision with available advice.

Improve this answer
3
  • 11
    While they haven’t technically ignored the remarks, for the purpose of the review it’s almost equivalent. Nothing happened and you have almost no idea why. The main difference is that it’s a little less impolite and you can exclude that it was accidental.
    – Wrzlprmft ♦
    Commented Sep 13, 2022 at 12:08
  • 3
    If the "changes are beyond the scope of this paper", this can be part of the replay to the reviewer, and sometimes it should also be mentioned in the paper itself. e.g. "Investigating foogle would be a more complete study of lampflow, but that is beyond the scope of this text."
    – Clumsy cat
    Commented Sep 13, 2022 at 14:12
  • If they have decided not to address the points, why are they bothering the reviewer with another round of review? That's the question.
    – Kaz
    Commented Sep 15, 2022 at 21:47
37

I had a very similar situation recently. I had, in my review, recommended including some additional experimental comparison to competing approaches, and questioned how one of the datasets was used in a different experiment. The editor, in summarising the reviews to the authors, had also recommended strengthening the experimental section by including, in some form, the comparisons I have suggested.

The paper came back after revision virtually unchanged, with the rebuttal stating that they do not think an exhaustive comparison to other approaches is needed, and providing some reasoning for using a dataset in a non-standard way.

I reviewed the paper as it was at that point, assuming no further changes will be made. This meant that I did not give the same recommendations twice (include X, change Y), but rather pointed out to the flaws in the paper caused by omitting X and Y, explained how I disagreed with the few justifications provided by the authors in the rebuttal, and explained why I think the paper is not fit for publication with these gaps. I concluded by recommending rejection.

I also sent a private response to the editor (as the system allowed it), to clarify that I have not re-made any previous suggestion that was explicitly refused by the authors, and am instead offering a review of the paper as-it.

Improve this answer
1
  • 4
    +1 I think this is the reasonable approach here.
    – Dilworth
    Commented Sep 13, 2022 at 14:42
13

I advise against declining the review. Good reviews are hard to get and cost a lot of time and effort. The editor wants to know what you specifically think of the revision. You can tell the editor what you think with very little effort. A new reviewer would have to start from scratch.

I would also focus more on your interaction with the editor rather than authors, at this point.

Similar to EarlGrey, I would think about how these points impact your final assessment of the paper. While it's frustrating if authors don't take constructive feedback, ultimately the decision will be whether to accept and what revisions to require. And the editor is relying on you to make recommendations. So if you think the paper is fatally flawed without these fixes, you can say that. Or if you think the paper is publishable but weak, or unpublishable.

Improve this answer
13

You’re overthinking this. There is no logical (as opposed to emotional) reason to decline the review, and given that you undertook to serve as the reviewer for the paper, it would be a bit irresponsible to refuse to see the process to the end.

If you think the paper should not be published in its present form, recommend rejection, and explain why. That’s all that is required of you after the detailed feedback you’ve provided. Your job is not to argue with authors, nor to write their papers for them.

Improve this answer
8

a detailed review report that I feel was not too harsh

it sounds like the cost-benefit of addressing the points you raised was negligible: if the report was not harsh, probably the authors read it as "the reviewer is suggesting nice things to be addressed, if only we had infinite time and resources ..."

Similar to the rain: you can walk and ignore it if it not too harsh, just addressing it by "I will get a bit wet, but I keep on walking" ... you would act differently if it was a harsh thunderstorm.

Improve this answer
4

I see two options here:

  • Briefly review the paper once more, noting in a clear way that the author did not fix most of the suggestions. Without giving a clear recommendation to accept or reject.

  • Briefly review the paper again, but indicate in a clear way that you recommend rejecting the paper due to it being not up to publication standard in the current state.

In general, I would lean towards the second option, because authors completely rejecting each and every suggestion of the review without any explanation on the reason they rejected it cannot be considered in general as participating in the peer review process in good faith. If you reject the majority of comments given, then you'd better explain what went wrong (if only to pay respect to the reviewer).

Improve this answer
1
  • 2
    I had a similar dilemma doing a recent review of mine (see my answer), and was considering sending in my feedback without including a decision recommendation. I then realised that the journal had contacted me specifically for my opinion when asking me to review (while the editor, and not me, has the specific duty of making a decision). I concluded that it would be rude to not provide what I had explicitly agreed to give -- my opinion (and decision recommendation) -- which the editor is then under no obligation to follow (if, for example, they consider my standards too high for their journal).
    – penelope
    Commented Sep 13, 2022 at 15:05
3

Many journals explicitly require point-by-point responses nowadays. I recommend to check whether the journal is question is one of them. If yes, it might even be that this is the only reason these responses were included at all – and thus are just a disrespectful and annoying way to cheat around this policy.

In particular if this applies but also otherwise, it may help to treat your situation very similar (but not identical) to the manuscript’s authors ignoring your comments completely, i.e., no change or reply. For the practical purpose of furthering a decision on the manuscript, this is no different: Nothing was improved and no arguments were provided. There are some nuances in politeness and ascertaining that your points were not accidentally missed, but those should not affect your recommendation anyway.

This point of view makes your decision clearer. If the authors had completely ignored every point in question, would …:

  • … you think that the editor wasted your time by sending you such a non-response? If yes, decline the review with this argument. Also consider not reviewing for the journal/editor altogether for some time.

    Here you have to take into account how obvious it is for the editor that the authors non-replied to many points. For an extreme example, I once had similar non-responses as a reviewer, but they were disguised by a page (!) of pointless waffling, such as a summary of the manuscript. I recommended to reject (and the editor agreed), but I don’t hold it against the editor that they didn’t spot this.

  • … reviewing whatever changes there are in the manuscript be a waste of your time? This applies for example if you would recommend to reject anyway, because a critical point of yours wasn’t addressed. If yes, decline to review with this argument, but offer to review the revision and response once it is done properly (if the editor would give the authors the chance). Mind that this is not very different from a very negative review. The main difference is that you do not bother to look at whatever changes were made.

  • … you feel neither of the above: Review as it is, and state that all your previous criticism still applies and you do not consider it addressed.

Improve this answer
1

It makes no sense to ignore the recommendations of a review, but then initiate another round of review to the same reviewer.

Is it not possible to communicate something along the following lines?

I acknowledge your decision not to act on the recommendations in my last review. My previous review round was thorough and I have no new recommendations. I wish you the best of luck with the paper!

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .