When reviewing a manuscript for the second time, is it fair to comment on aspects I did not realise the first time?

Question

I refereed a manuscript and recommended a major revision. Now, almost three years (!) later, the manuscript has come back to my desk. The editor told me to treat it as a revision and not as a new submission. However, since I don't remember the details, I'm re-reading the entire manuscript.

I'm noticing things that I consider need to be improved. That includes things that were identical three years ago, but that neither me nor the other referees commented on. There could be several reasons why I didn't comment on it the first time:

1. I had less knowledge and experience than I have now, or
2. the field has moved on since it's been so long, or
3. I simply didn't notice something I noticed now.

Is it fair to comment on it now, or did I lose my chance to suggest this improvement when I didn't suggest it in the first round?

On the one hand: I genuinely believe this needs to be improved.

On the other hand: if I were submitting a revised manuscript, I would not expect referees to comment on things they should have commented on the first time. If nobody comments on a particular paragraph, I would assume all approve.

Answer 1

Of course. Your duty is to the field, not to the author, or even to the journal. If you have noticed things that are wrong the second time, point them out. If they are stylistic, or extras, refrain. Lets say you found something wrong the first time, and then when you reread the paper you discovered it was not wrong at all, but the comment had been due to your misunderstanding. Do you think you have an obligation to stick by your original criticism? Of course not. Mistakes, positive or negative, of the reviewer deserve to be fixed when recognized.

Think of all the graduate students who may waste their time on the mistakes you did not point out the second time.

Also remember that you are providing advice to the editor. Thus the editor if they feel you are being unreasonable have the right to ignore your advice. That is not the a license to be unreasonable, but rather a perspective on your job.

Answer 2

I believe that it is entirely reasonable to make major requests of a manuscript on second review, as long as it is maintaining your basic standard for acceptance rather than moving the goalposts.

For example, I have had the experience where things about a paper were very unclear to me in the initial review because of shortcomings in the authors' presentation. Upon revision, those things became much clearer---but not in a good way. The extra information shed light on serious flaws in the authors' work, which has led me to in some cases recommend further major revision and in other cases recommend rejection.

On the other hand, some reviewers seem to like asking authors to do entirely new work, not because it is necessary for publication, but because the reviewer wants to know the answer or thinks it will make "a more interesting paper." This happens most frequently with "glamour" journals. I believe this sort of "moving the goalposts" on publication is not appropriate even in the first review, and doubly so in a revision review.

Answer 3

You certainly shouldn't refrain from suggesting improvements or corrections simply because it is the second round of refereeing. Go ahead and include them, as it can do no harm. It's also fine to point out minor things like typos or awkward sentences that you didn't catch the first time. And if you have just now discovered that the paper's main result is wrong -- or likely to be wrong -- you have an obligation to communicate that fact.

On the other hand, I would usually refrain at this stage from insisting on major new changes to the manuscript that I hadn't requested previously; especially things like

• A major reorganization or rewriting
• New experiments or analysis

If there have been new developments in the field that significantly affect the status of results in the paper, then it might be appropriate to insist on substantial rewriting or additional research, but otherwise I would say it was really your job to catch these issues the first time around.

Answer 4

If there are things genuinely wrong with the paper, you should definitely ask from them to be improved. If the issues are with things that could probably be handled better, yet which are not fatal errors, you might want to go easier on them. However, even in the latter case, there is no reason not to send back the paper for additional minor revisions before recommending acceptance.

When I review a paper with a lot of problems, yet which I feel could probably be published with major changes, I try to include an explicit statement in my first review that looks something like this: "At a minimum, the authors need to make the changes that I have suggested in order for this paper to be publishable. However, given the major problems that I have identified, it is not possible at this stage for me to judge definitively whether the paper's conclusions will be justified once the changes are made; it is possible that a heavily revised paper will still turn out not to be satisfactory." This makes it clear that I cannot adequately judge the correctness of the paper without the major changes being made. In my experience, such papers usually are publishable after the major changes are made (although often with another round of minor revisions), but it a significant number of cases, the revisions only serve to make clear that there are fundamental problems with the paper, and I ultimately have to recommend rejection.

Answer 5

On the one hand: I genuinely believe this needs to be improved.

That is the only relevant consideration here. Your job as referee is to provide honest, accurate feedback to the journal and the author about what conditions need to be met in order for the paper to be up to the journal's publication standards. There is simply no question of fairness here. Even if we ignore the 3 year delay, which seems like it was the authors' fault and no doubt helped create this situation, and imagine a scenario in which you as a referee simply made a mistake and overlooked something important in your original review, I would argue that as long as you haven't given your final seal of approval and recommended the paper's acceptance, it is not only your right, but actually your obligation to insist that the things you "genuinely believe need to be improved" to make the paper acceptable actually be improved. It may be a bit awkward and might cause some disappointment or hurt feelings on the part of the authors, but if you are acting in good faith I see nothing unfair about it - on the contrary, you are in fact acting to prevent a different kind of unfairness whereby a paper is accepted for publication despite having some major flaws.

I should add that such behavior may appear to bear some superficial resemblance to a "moving the goalposts" type of behavior, as alluded to in jakebeal's answer, but what you are thinking of doing is in fact different, since the moving the goalposts idiom implies someone acting in bad faith with the express purpose of preventing an adversary or subordinate from getting credit for attaining a goal. As Wikipedia says:

The term [moving the goalposts] is often used in business to imply bad faith on the part of those setting goals for others to meet, by arbitrarily making additional demands just as the initial ones are about to be met.

To summarize, yes, it is fair, and quite reasonable in my opinion, to comment on the aspects of the paper you did not notice the first time around, especially considering, but also independently of, the three year resubmission delay.