Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I refereed a manuscript and recommended a major revision. Now, almost three years (!) later, the manuscript has come back to my desk. The editor told me to treat it as a revision and not as a new submission. However, since I don't remember the details, I'm re-reading the entire manuscript.

I'm noticing things that I consider need to be improved. That includes things that were identical three years ago, but that neither me nor the other referees commented on. There could be several reasons why I didn't comment on it the first time:

  1. I had less knowledge and experience than I have now, or
  2. the field has moved on since it's been so long, or
  3. I simply didn't notice something I noticed now.

Is it fair to comment on it now, or did I lose my chance to suggest this improvement when I didn't suggest it in the first round?

On the one hand: I genuinely believe this needs to be improved.

On the other hand: if I were submitting a revised manuscript, I would not expect referees to comment on things they should have commented on the first time. If nobody comments on a particular paragraph, I would assume all approve.

share|improve this question
3  
Regarding reason (2), how do you avoid turning the process into an endless game of "catching up with the state of the art"? – Klaus Draeger Mar 4 at 17:59
18  
@KlausDraeger By not spending 3 years to resubmit the revision... in 3 months it wouldn't be so much of a problem (at least in my field). – gerrit Mar 4 at 18:24
    
@gerrit Exactly. It seems to be entirely the authors' fault that parts of their paper are three years out of date. They created the problem so it can't be unreasonable for you to ask them to fix it. (Well, if the delay was because of one of the other referees taking forever to write their report, that might be a more complex question. But I'd imagine most editors would have just given up on a referee who took much more than one year to review a paper, so that seems fairly unlikely.) – David Richerby Mar 4 at 23:19
up vote 8 down vote accepted

Of course. Your duty is to the field, not to the author, or even to the journal. If you have noticed things that are wrong the second time, point them out. If they are stylistic, or extras, refrain. Lets say you found something wrong the first time, and then when you reread the paper you discovered it was not wrong at all, but the comment had been due to your misunderstanding. Do you think you have an obligation to stick by your original criticism? Of course not. Mistakes, positive or negative, of the reviewer deserve to be fixed when recognized.

Think of all the graduate students who may waste their time on the mistakes you did not point out the second time.

Also remember that you are providing advice to the editor. Thus the editor if they feel you are being unreasonable have the right to ignore your advice. That is not the a license to be unreasonable, but rather a perspective on your job.

share|improve this answer

I believe that it is entirely reasonable to make major requests of a manuscript on second review, as long as it is maintaining your basic standard for acceptance rather than moving the goalposts.

For example, I have had the experience where things about a paper were very unclear to me in the initial review because of shortcomings in the authors' presentation. Upon revision, those things became much clearer---but not in a good way. The extra information shed light on serious flaws in the authors' work, which has led me to in some cases recommend further major revision and in other cases recommend rejection.

On the other hand, some reviewers seem to like asking authors to do entirely new work, not because it is necessary for publication, but because the reviewer wants to know the answer or thinks it will make "a more interesting paper." This happens most frequently with "glamour" journals. I believe this sort of "moving the goalposts" on publication is not appropriate even in the first review, and doubly so in a revision review.

share|improve this answer
    
I see you liked my grammar fix, but stuck with the scare quotes and triple-dash? Fair enough, but you could have given me credit for my efforts. – Aaron Hall Mar 6 at 20:43
    
@AaronHall The way that the review works, I don't know how to give "partial credit" in its mechanism. – jakebeal Mar 6 at 21:43

You certainly shouldn't refrain from suggesting improvements or corrections simply because it is the second round of refereeing. Go ahead and include them, as it can do no harm. It's also fine to point out minor things like typos or awkward sentences that you didn't catch the first time. And if you have just now discovered that the paper's main result is wrong -- or likely to be wrong -- you have an obligation to communicate that fact.

On the other hand, I would usually refrain at this stage from insisting on major new changes to the manuscript that I hadn't requested previously; especially things like

  • A major reorganization or rewriting
  • New experiments or analysis

If there have been new developments in the field that significantly affect the status of results in the paper, then it might be appropriate to insist on substantial rewriting or additional research, but otherwise I would say it was really your job to catch these issues the first time around.

share|improve this answer
2  
While this all makes a lot of sense, and the reviewer should have caught major problems the first time around, is it not still better for everybody that the reviewer resolves this problem by catching them the second time around rather than simply ignoring them? The desired end result is high quality science, not avoiding hurting people's feelings, no? – Lightness Races in Orbit Mar 5 at 23:23

If there are things genuinely wrong with the paper, you should definitely ask from them to be improved. If the issues are with things that could probably be handled better, yet which are not fatal errors, you might want to go easier on them. However, even in the latter case, there is no reason not to send back the paper for additional minor revisions before recommending acceptance.

When I review a paper with a lot of problems, yet which I feel could probably be published with major changes, I try to include an explicit statement in my first review that looks something like this: "At a minimum, the authors need to make the changes that I have suggested in order for this paper to be publishable. However, given the major problems that I have identified, it is not possible at this stage for me to judge definitively whether the paper's conclusions will be justified once the changes are made; it is possible that a heavily revised paper will still turn out not to be satisfactory." This makes it clear that I cannot adequately judge the correctness of the paper without the major changes being made. In my experience, such papers usually are publishable after the major changes are made (although often with another round of minor revisions), but it a significant number of cases, the revisions only serve to make clear that there are fundamental problems with the paper, and I ultimately have to recommend rejection.

share|improve this answer

On the one hand: I genuinely believe this needs to be improved.

That is the only relevant consideration here. Your job as referee is to provide honest, accurate feedback to the journal and the author about what conditions need to be met in order for the paper to be up to the journal's publication standards. There is simply no question of fairness here. Even if we ignore the 3 year delay, which seems like it was the authors' fault and no doubt helped create this situation, and imagine a scenario in which you as a referee simply made a mistake and overlooked something important in your original review, I would argue that as long as you haven't given your final seal of approval and recommended the paper's acceptance, it is not only your right, but actually your obligation to insist that the things you "genuinely believe need to be improved" to make the paper acceptable actually be improved. It may be a bit awkward and might cause some disappointment or hurt feelings on the part of the authors, but if you are acting in good faith I see nothing unfair about it - on the contrary, you are in fact acting to prevent a different kind of unfairness whereby a paper is accepted for publication despite having some major flaws.

I should add that such behavior may appear to bear some superficial resemblance to a "moving the goalposts" type of behavior, as alluded to in jakebeal's answer, but what you are thinking of doing is in fact different, since the moving the goalposts idiom implies someone acting in bad faith with the express purpose of preventing an adversary or subordinate from getting credit for attaining a goal. As Wikipedia says:

The term [moving the goalposts] is often used in business to imply bad faith on the part of those setting goals for others to meet, by arbitrarily making additional demands just as the initial ones are about to be met.

To summarize, yes, it is fair, and quite reasonable in my opinion, to comment on the aspects of the paper you did not notice the first time around, especially considering, but also independently of, the three year resubmission delay.

share|improve this answer
    
A quibble: moving the goalposts does not require bad faith. It also readily emerges from careless or self-centered actions, analogous to the "feature creep" usage listed in the same article that you link. – jakebeal Mar 6 at 16:31
    
@jakebeal I think it's arguable whether moving the goalposts could exist without bad faith. I'm not saying it necessarily can't, but feature creep is IMO a very different thing from moving the goalposts and would not be a good example, so I'm not quite convinced. Also, thefreedictionary defines moving the goalposts as "to change the rules in a situation in a way that is not fair, usually in order to make it more difficult for someone to achieve something," which also suggests the bad faith interpretation is the more common one. – Dan Romik Mar 6 at 17:41

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.