21

Earlier this year, I received a paper to referee for the top journal in my field of research. I know both of the authors. The first recently completed a Ph.D. working with a friend of mine. My faculty friend is the best-known person working in our area; he is an extremely careful and conscientious researcher, and working under him, the student produced some very good work. The other author of the manuscript I'm refereeing is the new post-doctoral supervisor of that recent Ph.D. grad. He is a very senior but not terribly distinguished scientist, and I think that some of his work is rather slipshod.

The first manuscript draft I got from this pair of authors had a glaring problem. There may have been a fundamental error in how they interpreted their results. There are two different ways that the system they are studying could behave, and they assume that it goes one way, apparently without even noticing the other. They might well be right about how the system behaves, but I sent back a report saying that the work would be publishable if they either explained why they only considered one of the two possibilities or expanded their analysis to cover both. The final conclusion is likely to unchanged whichever way the system behaves, but it is important to verify this; moreover, it is interesting in its own right to know which way things go.

After a rather brief period, I got a revised manuscript back from the journal. The authors had made a number of other minor corrections that I asked for, but they basically ignored my main point. At this point, I'm not sure what to do. I still feel like the paper contains a significant amount of interesting material, and it could be fixed without that much effort. However, I am upset that the authors made not effort to fix the actual problem I pointed out. They don't even really acknowledge the problem in their resubmission letter.

I was the only referee who turned in a report on the first draft, so what I say will almost certainly determine whether the paper is published. Should I send it back one more time, insisting that the changes I called for actually be made this time? Or should I recommend rejection, since the authors are apparently unwilling to make reasonable adjustments.

  • 1
    Is it well known that there are two possible ways in which the system could behave? Have you backed up your claim with references? Was your remark sufficiently strong to convey its importance? – Massimo Ortolano May 24 '15 at 2:18
  • 3
    I spend about two thirds of the report discussing the matter, so the authors must have willfully avoided responding. I suspect that the authors had not realized that different behaviors were possible when they wrote the paper. They made an assumption, based on how similar systems behave; however, the very features that made the more complicated system they decided to consider interesting also make it possible that it could behave differently. That might not have been obvious when they were thinking about the problem, but it occurred to me as an obvious question when I read the manuscript. – Buzz May 24 '15 at 12:36
  • 2
    I don't understand why you would do anything other than recommend rejection. Is there some reason why you would? – Ben Crowell May 24 '15 at 19:15
  • 2
    @Ben Crowell: The paper could still be a worthwhile contribution if they actually made the changes, which makes me hesitant to reject it out of hand. – Buzz May 25 '15 at 1:04
44

It's totally reasonable to be upset that the authors ignored your comments, but for the purposes of reviewing the paper, I think you should set that aside. I would let the editor make the decision as to whether the authors are "unwilling to make reasonable adjustments". Send him/her a note saying something like:

I mentioned in my initial report that it was essential for the revisions to address the issue of XXXX, but this has not been done, nor mentioned in the authors' comments. This paper has many good qualities, but I do not think the paper is suitable for publication if this issue is not addressed. If you feel it is worth the time to ask the authors for a further revision addressing this issue, I would be happy to read it and give my opinion.

I think it's up to the editor to judge between "this is a paper I really want in my journal, so I'm willing to put up with a little bit of crap" or "the authors are wasting our time, to hell with them".

  • 18
    @nateeldregdge: As an editor, I rather have reviewers come to the point than be circumspect. "This paper has many good qualities, but..." and "If you weel it is worth the time..." really don't adequately communicate what you think. Be blunt: "I believe that it is important that the authors address the issues raised in the first review, but they have failed to do so." It's the editors job to make a decision, your review does not need to point this out. – Wolfgang Bangerth May 25 '15 at 3:04
  • I find the response clear. It makes the expertise statement and makes clear that it is ultimately an editorial decision. – Captain Emacs Jun 6 '16 at 9:21
14

If it is indeed the case that:

  1. The potential flaw really could fundamentally change the conclusion of the paper, and
  2. The authors didn't even address your issue at all even in their response letter

then I would recommend rejection.

When doing a major revision, it is imperative that the authors at least speak to every point raised by the reviewers in the response letter, even if it is just to say: "No, we will not do this."

If an author blatantly fails to address a critical issue, then I feel that it is reasonable to conclude that they are avoiding the issue and recommend rejection. If the issue is important but not critical, or if you disagree with how the authors address it, then I think it is better instead to recommend major revision.

Even though you recommend rejection, if the editor wants to give them another chance, they can certainly do so. Similarly, even if they are rejected, they can amend their ways (or not) and go to another lower-level journal. So: vote your scientific conscience.

12

Should I send it back one more time, insisting that the changes I called for actually be made this time? Or should I recommend rejection, since the authors are apparently unwilling to make reasonable adjustments.

As long as they don't even react to the remark, the polite move would be to assume a good faith communication problem, especially given the amount of involved people. So I'd just ask for another revision and point out the remark was not addressed anyhow.

Personally, I'd avoid being subtle once they missed the point, so I'd just spell out that the authors should address the remark (in the paper), convince you they shouldn't in the resubmission letter, or you will recommend rejection. To avoid being too negative or aggressive, I'd also put in positive comments — say, you expect the point to be easily addressed.

  • 1
    I was already very explicit about what they ought to change. The only possibly conclusion is that the authors willfully ignored the main point of the review. – Buzz May 24 '15 at 13:02
  • 4
    @Buzz: or they didn't communicate well with each other on who should address the issue. Or the two "good authors" trusted the author you don't like to deal with it and he didn't. I've seen from close much more complicated screwups, so I won't try to cover all possibilities. – Blaisorblade May 24 '15 at 21:28
  • There were only two authors on this paper, the recent graduate and his new postdoctoral supervisor. I don't think poor communication between the two of them is likely, although extreme incompetence could always be an explanation. – Buzz May 25 '15 at 0:50
6

You should encourage the editor to get a second opinion. If you feel strongly, you should reject the paper again citing or repeating your comment about the alternate explanation. Based on what you've said here, it seems likely to be important.

  • I'd add an option of contacting them directly - if this has not yet been done. Seeing as both authors are apparently well known to the OP this might facilitate things. – Nox May 24 '15 at 1:23
  • 16
    @Nox: Assuming the journal uses anonymous refereeing, many people would consider it to be inappropriate for the OP to contact the authors directly (thus revealing his/her identity). See academia.stackexchange.com/questions/9523/… – Nate Eldredge May 24 '15 at 1:48
0

It seems to me that you should write a brief review which starts by saying something like "I recommend rejecting the submission X, on the grounds that the paper fails to even consider a plausible alternative interpretation of the experimental results, and thus the theoretical significance of the paper is indeterminate". In my review, I would reiterate the essential points about the alternative made in the earlier review, and I would also state this as an "I already mentioned this" summary. It is scientifically not acceptable to claim or imply that a specific conclusion has been established based on some evidence, when in fact multiple conclusions are consistent with that evidence.

It is not clear whether your previous recommendation was "revise and resubmit" or "reject" -- I assume it was "revise and resubmit", otherwise it would be strange for the paper to have been revised and resubmitted. I think therefore that it would be useful to explain to the authors and the editor the rationale for a more strongly negative recommendation, when the paper has not (apparently) gotten worse. A number of journals use the "one round of revision" standard, where you judge that after one revision, the paper is likely to be acceptable. Under that standard, if you judge that a paper is likely to be acceptable after one revision, it should be rejected. A corollary is that if a paper turns out to not be acceptable after a single revision, it should be rejected. Now, I do know of journals where 4 rounds of revisions are tolerated, so you may not be able to contextualize the "reject" decision based on announced journal policy, but you can certainly adopt such a standard, implicitly or explicitly, as your own quality standard for acceptance.

  • My first review was not a rejection, but it called for a major revision. The journal guidelines set a goal of determining publishability with at most two revisions. That gives time for major revisions, if necessary, then minor revisions that can be identified on a second round of reviewing. However, the review process can, at the editor's discretion, go on quite a bit longer. – Buzz May 24 '15 at 13:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.