Is there any research/study/survey/... that looked at to what extent reviewers are influenced by other reviews?

For example, ICLR 2019 published reviews publicly before all reviewers had written the reviews.

I am especially interested in the field of computer science (machine learning / natural language processing).

  • 6
    I am surprised that any place allows reviewers to see other people's reviews before they have submitted their own review - this seems to go against pretty much every bit of research I've ever seen on how group work of this kind should be conducted. Does this regularly occur in other venues? – BrianH Nov 6 '18 at 2:40
  • @BrianH AFAIK it's not common in my field (machine learning / natural language processing). – Franck Dernoncourt Nov 6 '18 at 2:41
  • 2
    @BrianH I don't know anywhere where this is the standard operating procedure, but I have received a small number of such review requests from journals. The editor would ask for a quick turnaround time after another referee dropped out late, and attached the previously completed report(s) for my "convenience". – Anyon Nov 6 '18 at 3:23
  • 2
    @BrianH I expect to see other reviewer's comments if I am reviewing a revision which is accompanied by the authors' response to reviewers. – Anonymous Physicist Nov 6 '18 at 6:15
  • 2
    Interestingly, ICLR 2019 does not let me see the other reviews before I submit mine, but if I log out I can see the other reviews first. – Thomas Nov 6 '18 at 6:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.