7

While I clearly encounter the fact that the preferred type/types of peer-review varies/vary across research disciplines, like pointed out in this question, I wonder which disciplines prefer which type/types and maybe even the underlying reasons.

Is there any kind of a peer-review map showing which disciplines prefer which type/types of peer-review and maybe even why?

  • 4
    I'm adding a CW answer so people can add their disciplines in the appropriate entry, and we can create such a map here. – Kimball Feb 21 at 14:02
5

Community answer:

Single blind

In single blind reviews, the authors are not given the identity of the reviewers, but the reviewers are given the identity of the authors. The editors handling the submission know the identities of everyone involved in the process, and the authors may or may not be given the identity of the editor handling their submission.

  • Pure Math
  • Physics (single blind is natural given the lively preprint culture)
  • Some parts of Computer Science:
    • Algorithms
    • Logic
    • Verification
  • Atmospheric Science / Meteorology (typically Open Single Blind, i.e. reviewer reports and author responses are openly available)
  • Library and information science

(if your discipline primarily uses single blind reviews, add your discipline here)

Double blind

In double blind reviews the authors are not given the identity of the reviewers, and the reviewers are not given the identities of the authors. The editors handling the submission know the identities of everyone involved in the process, and the authors may or may not be given the identity of the editor handling their submission.

  • Some parts of Computer Science:
    • Cryptography
    • Natural Language Processing
  • Economics (although preprints common in practice)
  • Public Policy/Administration

(if your discipline primarily uses double blind reviews, add your discipline here)

Triple blind

This is a system where the authors are not given the identity of reviewers, the reviewers are not given the identity of the authors, and the editors do not know the identity of the author and only find out if/when a paper is accepted (partial after-decision unblinding).

(if your discipline primarily uses triple blind reviews, add your discipline here)

Hybrid

Some disciplines use a hybrid system that is not easily classified as one of the above. If your discipline fits this description, note it below with a short explanation.

  • Human Computer Interaction (HCI) - examples: CHI and UIST
    • 2-3 "outside" reviewers review a submission double-blind. 1-2 reviews are single-blind reviews or meta-reviews by the equivalent of editors (members of a program committee), which both summarize the previously obtained external reviews and include the reviewers own judgement and discretion in assigning scores of their own. A light anonymization process is used, so blindness at any stage varies by submission. Final decisions are made collectively by a larger program committee, based heavily on the judgements of the editors as informed by external reviews, often using a score-cutoff (submissions receiving less than a certain score will not be discussed by the committee and are rejected automatically).

(if your discipline primarily uses such a hybrid system, add your discipline here)

Note that in all fields, how strict the blindness of submissions must be varies. Sometimes the submission must be scrubbed of any hint of who the authors might be, up to and including censoring citations that might suggest identity, rewording phrasing to speak of work in the third person, etc. Some venues within fields may even check to ensure that a search for the title of the submission does not return identifying results from previous talks or pre-print copies. Others take a light approach, and allow authors a great deal of discretion in choosing how heavily to censor their submission. Blindness does not mean that one party absolutely cannot figure out or determine who the authors might be, only that the identity of a party is not made intentionally explicit.

Shepherding

The Patterns Community in CS and other fields (including Poetry, actually) assign a shepherd to each paper. The shepherd works with the authors through several versions of the paper, seeking improvements. The shepherd reads a version and make suggestions. The authors produce a new version in light of the advice. Usually after about three rounds, the shepherd gives a recommendation to the conference committee.

If the paper is accepted by its shepherd and the committee then the resulting version of the paper will go to the conference, and be workshopped by peers of the authors in a session in which the authors take notes but, in the classic version, don't participate. A workshop isn't a presentation by the authors. The authors then write a final version of the paper that will appear in the proceedings.

Rejected papers might need to go through the process again.

There has been some evolution of the concept, but it is still used. If you read a paper in the proceedings of a patterns conference or in many books by community members it has gone through this process.

AFAIK, this process was first developed in the poets community where I assume it is still used.

  • 2
    @MichaelSchmidt There's a very perfunctory check on arXiv, but no review. People generally post it there, then later submit to a journal, say one of the Physical Review ones, which tend to use single blind review. As in, identity of reviewer is not known to authors. Things get less clearcut when you approach the boundaries to e.g. engineering / materials science / chemistry, but in my experience single blind certainly is the standard. And I'm in condensed matter physics (CMP). – Anyon Feb 21 at 14:42
  • 1
    @MichaelSchmidt What? How would triple blind even make sense? Are you maybe talking about the number of review rounds rather than "degree of blindness"? I currently have a paper in front of me that I'm reviewing for PRL, and it clearly includes the author names => it is a single blind review process. – Anyon Feb 21 at 15:01
  • 4
    I believe Triple blind would be where the editors also do not know author identity until after completing their work. – Dawn Feb 21 at 15:21
  • 1
    @BrianH If there is a field where "reviewers don't know who the author is", but the authors know who the reviewers are, then I think the two categories should be distinguished. (I don't know of one myself.) If not, as highlighted by the discussion in the comments, it could still be useful to define the terms in the answer or question. – Anyon Feb 21 at 16:36
  • 3
    I've added term definitions, as well as a note about the strictness of blindness as this varies quite a bit between fields and is not covered by the blindness categories alone. Please do edit if anything added is not more or less universally correct across fields. – BrianH Feb 21 at 17:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.