31

For the first time, I had to review a revision of an academic paper (I have already reviewed several other papers, but they were all rejected after the first round). While the authors had clearly improved the paper (after the first round, major revisions were requested by the editor (and myself)), several major issues mentioned in the first round of reviews were still not (or not correctly) addressed. Furthermore, due to the improvement in writing and structure, the paper was easier to read, and I was able to identify several flaws or strong limitations that I did not report in the first round. Consequently, I, again, recommended major revisions (while I hesitated with rejection due to the major flaws).

After having submitted my review, I received the reviews of the two others reviewers. To my surprise, both were positive (1 line comment such as "Issues have been addressed and I have no further comment") and the editor requested minor revisions.

In view of this, I am wondering if I have misunderstood how I should review a paper after revision as I did not make a real difference between round 1 and 2. Therefore my question is how far can/should we go in reviewing a paper? If at each round, improvements are made but new flaws or mistakes are spotted, should we stop mentioning them at some point? This process might be theoretically infinite. Furthermore, as an author, I know that several rounds of reviews are exhausting and stressful so I do not want to be "that reviewer" (the one who is too picky and who completely slow down the publication process).

Improve this question
0

5 Answers 5

Reset to default
23

(1) The comments of other reviewers should not bother you; often reviewers will be from mutually exclusive fields, so as to provide better representation. You should stand by your professional opinion irrespective of the others.

(2) I firmly believe that you should spend a lot of time on first and second review. If you are finding new and significant errors beyond these rounds, it means you didn't do your job rigorously before.

EDIT: Several comments highlight the valid point that sometimes revisions are required just to make the paper readable. My position on this is that if more than 2 'readability-reviews' are required, the authors are either unable to use the language adequately (and should be directed to a suitable resource- a reviewer is meant to check technical content, not be a style coach), or are unwilling to make these suggestions; in which case there is no point asking once again.

Comments/questions regarding your previous objections are ok, but entirely new objections in third or subsequent rounds leave a bitter taste and lead to a lot if time being wasted.

Improve this answer
8
  • 2
    Actually, regarding 2, it may mean the paper should have been sent back to the authors before round 1. In my experience, this happens because the initial paper is inexcusably sloppy, but the editor does not want to offend the authors.
    – Dawn
    Commented Jun 19, 2018 at 19:09
  • 11
    Well, changes made in light of earlier objections may create new problems. This is not necessarily the reviwer's fault.
    – cfr
    Commented Jun 20, 2018 at 2:36
  • 7
    I've also had the experience of not seeing serious problems in earlier rounds because the unclear writing masked these issues. That is a fundamental part of OP's question. Your answer seems to imply that OP did not spend sufficient time in earlier rounds, which I don't think should be assumed.
    – Tripartio
    Commented Jun 20, 2018 at 6:09
  • 1
    As a counter to your second point: I have often reviewed papers that were so sloppily written that mistakes could only be spotted upon second or third rewrites. If I had to catch everything in the first or second round, I'd essentially be rewriting the whole paper.
    – Designerpot
    Commented Jun 20, 2018 at 7:21
  • @Dawn - I agree. Have edited the answer to reflect this. Nevertheless, I don't believe this should be done more than once- we are wasting everybody's time by doing so.
    – AppliedAcademic
    Commented Jun 20, 2018 at 8:41
12

This has also happened to me, so I appreciate this question. I think this situation occurs, in large part, because a totally sloppy first draft is sent out for review rather than desk rejected. So in some sense this is not your fault.

In the end, I gave a review including all the issues I had identified and then asked the editor for feedback regarding my (second and third round) reviews. He explained that he was pleased with my approach, although in the end he ended up telling the authors that some of my comments from the third round were optional.

The feedback I received makes me believe that a reviewer should continue to point out issues in the second and third rounds, even if they are new. Then alert the editor to the fact that you are finding new issues (perhaps note that it is due to the substantial rewriting). Final decisions are up to the editor and usually they are happy to be able to consider the points, even if (in the end) they tell the author that some points can be ignored at the later stage.

Improve this answer
9

I appreciate the fact that you are asking this questions with sensitivity to the authors' feelings. Although we should be professional and objective in doing our reviews, we should also respond in ways that take the authors' human feelings into account.

In this case, I think you should indeed present the new concerns you have at each round, especially if they were hidden earlier do to poor writing. However, when you do so, I recommend that you send a note to the editor explaining your concern that it might be unfair to the authors to bring up new major issues so late, and so ask the editor to use their discretion in making the recommendation. That way, you fulfill your responsibility to objectively identify the problems with the article, but you absolve your conscience of any possible guilt for giving the authors an unfair experience--it is the editor's responsibility to make the decision, not yours.

Improve this answer
3
  • 10
    If a paper has problems from a scientific point of view, then there shouldn't be consideration given to the result of the review being "unfair" to the authors (as long as it's not "unfair" due to bias). The point of reviews is so that we, collectively, do good science. We shouldn't sacrifice good science just because meeting that standard is inconvenient for, or "unfair" to, the authors. OTOH, it is quite reasonable for the OP to recognize that the author's might be frustrated and A) preface their comments with wording meant to reduce the emotional impact of the review and
    – Makyen
    Commented Jun 20, 2018 at 7:53
  • 2
    B) introspectively attempt to determine what they could do better/different to have more issues resolved/identified in the first review, so that such situations are at reduced/eliminated in the future. Basically, the point I'm trying to make is that you can be considerate of others/the feelings of others and put effort into making them not feel as bad, but when doing so you shouldn't sacrifice the primary goal of the process: good science.
    – Makyen
    Commented Jun 20, 2018 at 7:54
  • @Makyen, good points. I agree.
    – Tripartio
    Commented Jun 20, 2018 at 10:28
8

Stand on what you believe is most correct and do not leave the paper with flaws like that. Your decision matters before it is published. After publication it would remain online and others may not be able to figure out these flaws. In fact others may follow these errors and publish papers in the future. Your obligation to the paper takes precedence over that of the journal i.e if the paper is published elsewhere and the comments are well addressed it is better than publishing it in the journal with those flaws you have identified. However, it is important to mention that you should try at the first few rounds to read the paper thoroughly.

  • Give every detail and proof you have concerning the paper while being polite to the authors. Instead of saying 'the authors' say 'the paper' etc
  • Apologize and make it clear that you have not seen these issues in the previous round.

The point is that the reviewer is like a Judge, whenever he finds new evidence, his judgement could change

Improve this answer
2

Thank you for raising this question. In my view, (new) valid and constructive issues can always be brought up, irrespectively of the number of review rounds. Authors will appreciate the help for improvement.

However, me and my co-authors received three pages of new issues after a major revision and acceptance by two reviewers. These issues, in our opinion, were non-coherent, far from constructive, some factual wrong (but stated as facts) and we subscribe them to a lack of knowledge of the reviewer in this field. We rebutted most of the issues as we do not consider these as improvements.

Because of the quality of the review, we consider these new issues as inappropriate. Otherwise we would have welcomed new issues.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .