63

A few months back I received a request from a reputed journal to review an article. The article was in a fast-track mode because of its assumed importance. I did review it, and found that it makes some improvement over the existing models, and hence does not rate high on the novelty axis. Moreover, there were serious problems in the article both conceptual and mathematical. Lastly, the English used was erroneous. I wrote these things to the editor, and he made it a regular article and asked the authors to revise it.

They did, and the manuscript came back to me. Unfortunately, the mathematical problems were simply ignored by them, and only cosmetic changes were done. I wrote back to the editor about this, and requested him to ask another review with all the clarifications/modifications. But after that journal did not communicate with me. Today, I was surprised to see that the paper has already been published! I went through it, and found that some minor changes that I had suggested (like changes to a figure to make it more informative) have been done, but the main issues that I had raised have been pushed under the carpet. I find this to be a complete insult to the reviewer, and also a kind of scientific dishonesty. I thought of writing to the editor, but I just want to know if something like this is regular, and if it has happened to anyone. If yes, how should I proceed from here? Just shut my mouth and carry on?

  • 4
    Just to put the shoe on the other foot for a moment: how would you have preferred the editor to act if you were the author of said paper? – nick012000 Aug 28 at 4:05
  • 24
    @nick012000 : I would have made the changes or replied to the reviewer justifying my work. But unless the reviewer is convinced, how could the work be published, especially when the objections are serious? – Peaceful Aug 28 at 4:07
  • 5
    Is the journal one that allows people to publish comments on or responses to other people’s work? – nick012000 Aug 28 at 4:13
  • 4
    If you don’t want to work with that journal again, you could also include the name of the editor when publishing your comments ie that they were told prior to publishing... – Solar Mike Aug 28 at 4:19
  • 7
    Maybe the editor asked another reviewer for an opinion as well. – usul Aug 29 at 0:21
72

This probably isn't something to fight over. Possibilities abound:

  • Perhaps you misunderstood something.
  • Perhaps the other reviewers were positive on the paper.
  • Perhaps the authors provided arguments that your rejection reasons aren't applicable, and the editor found them convincing.
  • Perhaps the editor thinks it's better to accept a potentially bad paper than to reject a potentially good one.
  • Perhaps the authors said they cannot fix the issues (e.g. funding ran out, one of the lead experimenters graduated and is no longer in the group, etc) and the editor made the judgment call to accept anyway.
  • Perhaps the journal is short on papers to fill its issues and so is accepting borderline papers.
  • Perhaps the editor simply made a mistake, but since the paper is already accepted, decided to stick with accept instead of rescind the decision.

Ultimately journals are going to publish whatever their editors think are acceptable. Reviewers do not "give permission" to publish something; they only offer recommendations. In the same way if there is a backlash against the journal for publishing this paper, it's the editors who take the heat, not the reviewers. So even if your objections are correct, it's probably still not something to fight over.

If it really bothers you, you could email the editor asking why they accepted the article in spite of your comments. If the response they give isn't satisfactory, you could refuse to review for and/or publish in this journal in the future. If it really bothers you and you feel taking retributive action against the journal is justified, you could try denouncing the paper on social media (high-level summary of what happened), but be psychologically prepared for the drama that might follow.

Alternatively, you could view the entire episode positively - hey, I can now write a paper arguing why this paper is wrong!

  • 3
    I do agree. But I would like to point out that so far whenever I reviewed an article, whether accepted or rejected, the editor has always informed me so. The fact that this time it didn't happen when I questioned the core of the paper makes me feel bad. Editor might have simply said that the paper is now fine and we accept it. Period. – Peaceful Aug 28 at 7:09
  • 1
    @Peaceful does the journal use an editorial management system? If so, there's a chance that you weren't informed because of technicalities: i.e., you didn't review the last version of the paper that was accepted or rejected. I can't explain why this system is there, but it was in Editorial Manager when I last used it. – Allure Aug 28 at 7:16
  • 2
    Basically if paper is submitted -> editor invites reviewers (including you) -> paper is sent for revision (you are notified) -> paper is revised -> editor invites reviewers, but does not invite you -> paper is accepted, then you aren't notified about a final decision because you didn't review the revision. If this happened, I'd guess that the editor didn't invite you the second time because he's already made a decision that went against your judgment. – Allure Aug 28 at 7:18
  • 6
    If you're wondering why the editor might not invite you if he's already made a decision that went against your judgment, I once had a reviewer who wrote something like "this paper should still be rejected, however, I note you have already decided in its favor so I don't understand why you're asking me". – Allure Aug 28 at 7:22
  • 1
    "Alternatively, you could view the entire episode positively - hey, I can now write a paper arguing why this paper is wrong!". Very good point! – MxNx Aug 30 at 10:01
74

EDIT: The question has been changed since this was written.

Your key misconception is that the editor needs reviewers' permission to publish a paper. Actually, the decision to publish rests solely with the editor.

In this case, you disagree with the editor, but we do not have enough information to tell who is correct. If you think the errors in the paper are important, then once the paper is published you may be able to submit a comment to the journal. Do not do that before the paper is published, because for most journals you must maintain the confidentiality of peer review.

  • 2
    Probably I shouldn't have used the word "permission"; I know that the editor doesn't need it. However, the problems are "serious". What is the problem in convincing the referee with solid arguments instead of simply ignoring him/her? – Peaceful Aug 28 at 7:08
  • 13
    @Peaceful if the editor disagrees with the referee, then convincing the referee with solid arguments takes time and effort but serves no useful purpose. It would be polite (but not absolutely necessary) to inform the referee of the decision, and it makes sense to ask questions to referee if their position is unclear; but if the arguments made by referee are clear to the editor but the editor intentionally chooses to act contrary to their recommendations, then why should they spend their time arguing with someone? As you say, the editor doesn't need your permission, that's their choice to make. – Peteris Aug 28 at 13:08
  • 12
    Any reasonably academic who cares for intellectual honesty should argue with anyone that raises serious scientific objections about a work they are editing or involved in. That is basic. At least the editor should note the referee who contributed freely to assist with the reviewing process. Indeed, it is common that editors are busy and choose the easy road of ignoring the referees. But let us not justify such intellectual laziness. – Dilworth Aug 28 at 18:50
  • 4
    +1 The decision to accept or reject a paper lies entirely with the editor(s). All a referee can do is offer a recommendation. I've even had papers get rejected after receiving glowing referee reports. (This has happened to me both as the author and as the referee.) Moreover, does the OP even know for certain that they were the only referee? In my field top tier journals often send papers to multiple referees, as do editors attempting to fast track a paper. – Ben Linowitz Aug 28 at 21:06
  • 4
    I had to read the first sentence a few times to figure out what you meant. I think if you say "Your misconception is X" the standard interpretation is that "you think X but it is wrong" and not "you do not realize X," or is this just me? – Kimball Aug 29 at 2:18
8

tl;dr: You may want to send an email to the editor about this. Although it will not change anything about this paper, it will be a good lesson for the editor to engage in more constructive correspondence with referees who are spending a lot of effort for free to enable the existence of the journal itself, as well as to be more responsible in his/her decisions next time.

--

This is something that can and does happen in reputable and even prestigious journals. We do not have the full picture, only what you wrote, but based on my experience, I tend to believe you are correct, and the editor made a judgment that takes into account non-scientific factors such as:

  • Efficiency. He/She does not have time to deal with the details too much. He/she needs to make a fast decision and it's safest to let the paper in because it was on a fast pace track anyway.

  • Politics/Importance/Perceived-importance of papers/author. Since it was on a fast track the paper was probably important for some reason to the journal/editorial board. Maybe it gives them some prestige? Or whatever reason. The editor knew there is a reason for concern, but went with publishing it because he/she decided to ignore what they perceive as "details" that "do not take into account the whole picture", or something like that.

  • Possibly, the editor did a genuine decision, believing the paper merits acceptance, and that "you are just picking on the details". They may have a different view than yours, they may think that details are unimportant.

Overall, I tend to agree with your view: details are extremely important and decisions should be made based on objective merits solely as much as possible. Unfortunately, that is not how the system works.

Conclusions: You may want to fight a bit over it. I don't see it as harmful. Simply send an email to the editor to inform them that you think they made a wrong decision as long as the reviewers don't address your concerns. This will not change anything for the present paper, but for the next paper this editor will be more cautious I assume. He/she may be a bit pissed off by your email, but so be it.

5

There is a slightly different scenario in which it may be useful to contact the editor, which is if it appears that your review was never even sent to the authors.

I know someone who reviewed a paper, she sent the review, but when the paper was accepted there was no evidence they had taken notice of it at all. She contacted the editor, who then found out that her review had never been sent to the authors in the first place. The editor profusely apologised to all parties involved, retracted the acceptance, asked the authors to revise taking into account her review, which they then did.

4

As others have already said, it is up to the editor to decide if a paper gets published or not. That being said, I have been in the same situation before, and to be honest, it is extremely annoying to carefully write a review - I usually put more care into reviews where I recommend rejection than reviews of good papers - only to have it dismissed by the editor.

My solution to the problem was simple. I stopped accepting referee requests from that journal, and submit my own work elsewhere. Since it is a well known mid-tier journal in my field, collaborators ask me why I don't want to submit there, and I tell them the story.

  • I don't even think there would be a problem with submitting your own work there. The journal is inviting submissions, you're the boss of how you decide to allocate your reviewing time -- there's no genuine problem with submitting in a journal you wouldn't want to review for. – a3nm Aug 29 at 21:41
  • 1
    The reason I don't submit there, is simply because I will rather not publish somewhere that a) does not take referee time seriously, and b) publish what I this are sub-par articles. – nabla Aug 30 at 20:00
3

This isn't something to fight over. The editor made an editorial decision based upon your review, other reviews, and whatever the editorial policies and goals of the journal are. This is the proper editorial role.

As to what you can do -- you have the option of doing nothing, you can counter the problems in the paper in a paper of your own, if it's appropriate to do so, you can write a letter to the editor (I'd recommend doing this in the role of a reader of the paper, and not a referee, which would be inappropriate) -- in other words, you would do exactly what you would do if you had nothing to do with the reviewing process and read a paper you had issues with.

Lastly, if you were really offended by the process, you might consider whether or not you'd accept review request from that journal or that particular editor ever again. There's certainly a "cut-off-your-nose-to-spite-your-face" aspect to effectively taking your ball and going home that might negatively impact you in the long run to turning down reviews, especially if you make the reason for that action known to the editor, but it might help you make your point.

  • 2
    Why not voice your concerns with the review in the role of the referee? This might get the most complete explanation from the editor (for example that other referees had a different stance on it) – eckes Aug 28 at 23:33
  • @eckes -- IMO, the editor simply does not owe the reviewer an explanation. Pursuing correction/explanation/satisfaction through the review process will be less than fruitful. Push comes to shove, it makes very little difference whether the other reviewers had a different stance on it, or if the editor overruled all the referees, or whether the editor generated a random number to come up with a decision. Requesting more information about the editorial decision is something for authors who have been rejected do, not something referees do. – Scott Seidman Aug 29 at 12:20
  • 2
    I think an editor and a publication owe someone who is doing work for free respect and transparency. – eckes Aug 29 at 12:32
  • @eckes yes, an editor does owe the referee respect, and will probably answer such a request very politely -- though it remains pointless. The referee has already voiced his concerns about the paper -- in the original review. Perhaps the concerns were not expressed clearly enough. That would be on the referee, who can use the experience to better frame reviews, if this is the case. Most journals (including all the journals I've ever reviewed for) provide all the referees with all the reviews, and if this isn't the case with this journal, requesting the reviews is certainly not out of line). – Scott Seidman Aug 29 at 12:47
  • Why is it pointless? Obviously the OP cares about the effort they invested in the review that the journal requested of them -- the journal don't owe them explanations, but acknowledging the effort isn't pointless, it's just basic decency. – a3nm Aug 29 at 21:57
1

Although it is not specifically helpful here, remember that publishing is only the first stage in peer-review, broadly conceived.

You've done due diligence trying to fix things before they make it to the broader world (perhaps you should have argued for rejection initially?). But this isn't the only poorly done science out there: try not to amplify those signals by being careful about what you cite.

Ideally, every paper is perfect and sound in all ways. But, given a flawed paper, there is also a question of whether some subset of the paper --- the core idea, perhaps --- might have value to the community. What you saw is disqualifying mathematical issues may not have been the part of the work the editor saw as valuable.

1

I don't think it's worth fighting over this, in the sense that it's the editor's choice to disregard your comments and publish the article. And I don't think you should care -- if the article is bogus, it will be the author and the journal's problem, not yours.

I do think it is very disrespectful to ask for someone's time to review something, and then ignore their feedback without keeping them posted. No matter the merits of your comments, no matter all the good reasons there might be to overturn them, at the very least the journal should have informed you about their choice of publishing the article nevertheless. Sadly it is pretty common that reviewers don't hear back after sending reviews, but this does not make it acceptable.

If you feel upset about this, I would suggest writing to the editor and express your surprise about not being informed about the outcome of the feedback that you sent. I wouldn't argue about the article's quality (it's probably too late to debate this) but simply ask whether they believe it is OK to take someone's feedback and summarily ignore it. This gives the journal a chance to clear up possible problems (e.g., forgot to send your comments to the author, forgot to inform you, etc.), and to apologize if they screwed up.

And if you are not satisfied with the journal's handling of this, you can simply decline further invitations to review for them -- or even block them and not bother replying to them. Reviewing is volunteer work, so you are free to decide how and for whom you want to do it. Some people in your research community might pressure you about doing reviews for your community's venues, especially if you submit your work for publication there (see e.g. this question), so you might need to hold your ground. In my opinion, though, this is bogus, and allocating your review time to the right venues is a completely legitimate way to push academic practices in the right direction.

0

As a reviewer, you are an advisor to the editor. You are NOT a gatekeeper. You are not the editor. Not the author of the paper. As an author, I may refuse to make a change that you advise. It is then up to the editor to decide if he will side with me or with you.

Even if you think you are right, you still have to allow the editor to make his own decisions. And realize some percentage of the time, you will disagree with them. It is his magazine. And it's the author's byline.

I advise not to get to over-invested in a particular publication incident. There are some questionable papers published. And there are some good papers that get stifled. It's not a perfect funnel.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.