25

There have been a number of questions asked here, such as this question, or this question. The first question asked about how to find the authors of peer reviews. The second discussed the benefits of anonymity of peer review. However, I have been unable to find a question address the antecedent issue: why is peer review anonymous?

What are the historical reasons behind this practice? Was peer review always anonymous? What are the benefits behind anonymous peer reviews? Has the academic community learned from previous experiences that anonymous peer review is the best sort of process?

It also seems like a few journals as analyzed in this article (https://doi.org/10.1038/s41467-018-08250-2) performed open peer review with a sizable proportion of authorship of peer reviews also providing their identities (8.1%). In this paper the authors suggested that anonymity protects individuals from retaliation. It would seem to me that if all of a sudden all peer review were public, attempts at retaliation would be easy to identify and address. However, with the current state of the review process, one could sometimes guess at the author of a negative review and secretly retaliate. How has the academic community overall arrived at general acceptance of anonymous peer review?

15
  • 16
    "attempts at retaliation would be easy to identify and address" Not necessarily. There is usually a subjective component to assessing the merit of a paper, and it might be impossible to determine whether a particular negative review has roots in personal animosity. Apr 22, 2020 at 7:20
  • 5
    "with the current state of the review process, one could sometimes guess at the author of a negative review and secretly retaliate." That seems to be the exception rather than the rule, see the related question. Apr 22, 2020 at 7:22
  • 2
    @Viktor It's absolutely likely that such things happen, but the total amount of retaliation taking place is probably much higher if identifies are generally known. Apr 22, 2020 at 7:26
  • 2
    The sample of reviewers choosing to publish their name might be very far from random. The only times I've had reviewers reveal their name to me was when they were very well established already, safely tenured and with a good reputation, while I was a PhD student. In other words, they have no retaliation to fear. This would be an interesting question itself, but probably hard to answer without unpublished data.
    – gerrit
    Apr 22, 2020 at 7:54
  • 4
    In this day and age of misinformation, hate, and anonymous attacks on social media, why would one think there would not be similar retaliation for bad reviews?
    – Jon Custer
    Apr 22, 2020 at 13:18

3 Answers 3

48

The big reason for anonymous peer review is to allow the reviewers to speak their mind without fear of the authors taking revenge. Even if it's easy to identify, retaliation can happen: it's already decently common for authors to get angry when their paper is rejected for reasons they do not agree with (example). Even if the authors don't retaliate this obviously, future relations with the reviewer can easily be strained: for example, say the author and reviewer communicate several times and eventually the author decides that the reviewer's an idiot and submits elsewhere. If the author later finds themselves reviewing the reviewer's grant proposal, it will be hard for the author to remain objective.

Another issue with non-anonymous peer review is, the signature of a senior person as a reviewer can carry the implied threat of "do what I say or I will crush you". The consequences can be unpleasant for the authors, especially if they're junior. Example.

Yet another issue with non-anonymous peer review is that it's not clearly advantageous. There is no strong evidence that open peer review improves review quality (or makes it worse for that matter). Instead there was an immediate negative effect in that more people decline to review. If it's the case that there's no advantage, then implementing a fix seems rather unnecessary.

5
  • 3
    Could you explain your second example in a bit more detail? I do not understand how this is an example of “do what I say or I will crush you”. Thank you for bringing out your other points.
    – Viktor
    Apr 22, 2020 at 18:58
  • 3
    As an addendum, anonymity is normal in most circumstances where it's important for the commenter to be able to speak their mind completely openly and honestly - for instance, that's why the Aviation Safety Reporting System operates almost entirely on anonymous reports, and why police departments have anonymous-tip lines.
    – Vikki
    Apr 22, 2020 at 19:36
  • 4
    nevermind applying to join the department of an enemy referee: much more damage can be done at the grant application level, where retaliation can be fatal to a grant proposal. Apr 22, 2020 at 21:59
  • 2
    @Viktor the link came from the answer to another question: academia.stackexchange.com/a/122462/84834. As JoshuaZ wrote in a comment there, The concern is in part that if someone is sufficiently prominent in a field they are by non-anonymizing their review potentially engaging in implied intimidation. In this case Gallup (who pioneered the field) signed his reviews, the editor did not anonymize, and the author (Jordan) interpreted them as "violently anti".
    – Allure
    Apr 22, 2020 at 23:41
  • To paraphrase @Allure's second point: Many reviewers write frivolous, careless, superficial reviews. But - it's quite likely they would continue to do that even if their names were on those reviews.
    – einpoklum
    Apr 24, 2020 at 7:33
14

If I review some professor's science paper and it's poorly controlled garbage, I either accept it anyway or I have a new enemy. Now I have to deal with the fact that submitting papers to all the journals that they're editors for will go poorly for me, and submitting grants that they'd be part of study sections for will be a waste of time for me.

Even people who don't have to worry about their own careers need to worry about this, because big name scientists fight by trashing each others' students' careers. Do you really want to have some promising junior scientist who worked for you get repeatedly shit on because you reviewed a paper?

5
  • 2
    I have two concerns about your last point. If all peer review were signed you would be able to clearly establish to your field that this scientist who is trashing your students is selfish and not objective. If the harsh remarks are objectively true, than does this not improve the quality of peer review? Lastly, what prevents a reviewer who wants to be the dominant scientist in a field from “trashing” the careers of the students of their competitors? In a single blind the applicant does not know their reviewer, but the applicant is known by the reviewer...
    – Viktor
    Apr 22, 2020 at 18:55
  • 1
    @Viktor Nothing stops that and it's a thing that happens a lot. It's mitigated by having editors who aren't anonymous and being able to put names forward for reviewers that you think will be fair to you. Establishing some senior scientist is selfish and not objective might make you feel nice but really won't matter to anyone even a little bit. Having some senior scientist as your personal enemy will matter a lot (but only to you; nobody else will care aside from maybe some sympathetic noises over drinks).
    – user120011
    Apr 22, 2020 at 19:37
  • I think it would matter to government funding. Especially in some political climates in some countries. I agree with your general point about the scientific community.
    – Viktor
    Apr 22, 2020 at 19:45
  • as an addition to my above point. Certain legally prohibited reasons for rejection may not be evident from a single peer review but a pattern of conduct could be demonstrated. I see a lot of potential benefits, but I also understand the potential downsides.
    – Viktor
    Apr 22, 2020 at 19:55
  • 2
    @Viktor "If the harsh remarks are objectively true" - and if not? Even "experts" may have biases, preferred ways of doing things, or simply are stuck in their own little field. There is no "objective" here. Knowing that, as a reviewer I always try to bring the best out in the given paper, given the authors' premises - despite that, I am also quite aware of my own biases and that they may still sway me. I am also aware that other reviewers' biases are not always as meticulously counteracted. Apr 23, 2020 at 10:43
4

Without scratching below the surface it seems anonymous peer review is the only way to maintain standards of quality and honesty. Not everyone who is a qualified reviewer is going to be 100% comfortable being openly critical of their peers' work; if the integrity of the results suffers even slightly over time due to that it is reason enough. On the flip side, the reviewers without any qualms about brutal honesty will see less difference btwn sugar coating and tactful delivery, which may result in an unfavorable imbalance towards more harshly toned reviews on average, and thus potentially more discouragement than encouragement to the reviewees.

10
  • 2
    “it seems anonymous peer review is the only way to maintain standards of quality and honesty” — citation definitely needed. This is an extremely contentious point, and not at all self-evident. Apr 22, 2020 at 16:41
  • 1
    Sorry I guess this wasn't clear but "without scratching below the surface" ~ "off the top of my head" and also "I haven't read any journal articles on the topic." An original idea, so in lieu of citations I'll explain my thought process and you tell me what's contentious. I do appreciate the irony of the label you slapped on along with the demand for evidentiary support without even a semblance of the same on your own part, hehe. Will be back in a few minutes with my defense. ;-)
    – AmishNick
    Apr 22, 2020 at 17:10
  • 2
    I mean, you were making the claims, the onus for evidence is squarely on you, not me. That’s how this works. Anyway, if you don’t believe me I suggest researching the topic of open peer review. Note that I’m not making any claim regarding the quality of evidence in favour of this, merely that the debate exists. By contrast, your claim is extraordinarily strong (and stronger than that in other answers), by saying that this is “the only way to maintain standards of quality and honesty”. As far as I’m aware there’s no evidence for this whatsoever. Apr 22, 2020 at 17:14
  • 5
    "The debate exists" about the earth being round. Using that as a standard is not intellectually honest. Every open-review journal that I'm aware of has withered and died and closed-review journals are all doing fine. That seems like pretty good evidence to me.
    – user120011
    Apr 22, 2020 at 18:03
  • 1
    (1) Without scratching below the surface it seems anonymous peer review is the only way to maintain standards of quality and honesty. Technically I should have said “it seems to me;” I thought that was obvious though. If I knew was treading on such contentious grounds I would've worded this to stand up better to attack, oh well. My opinion is not a matter of contention, but if you insist on taking this literally then the entire rest of this post I’m writing now is a waste of time since it’s literally only purpose is to prove an inconsequential argument.
    – AmishNick
    Apr 22, 2020 at 18:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .