I have recently accepted to review for a machine learning conference. It's a conference with chairs, area chairs (AC), and then reviewers who are managed by the AC's.
This conference is double-blind, with a rebuttal phase, and what follows was never mentioned in the invitation to review, the review instructions, nor on the website of the conference. But regardless of this, I would like to understand what could be the rationale for it:
- reviewers are assigned papers. Each paper is assigned to an AC, and reviewed by 3-4 different reviewers. Reviewers do not know the identity of the authors.
- each reviewer sends their reviews
- the authors get the reviews (do not know the reviewers' identities, nor the AC's), and write a rebuttal
- the reviewers are given the rebuttal for the paper (still do not see the authors' identities), and see the other reviews. Moreover, they now also see the identities of the other reviewers for the paper. (The authors still of course do not know these.)
- the reviewers (managed by the AC, whose identity is also known to the reviewers, discuss and update their reviews in light of the rebuttal and the other reviewers' reviews and comments.
Now, I do not understand the point of the part I emphasized, and it seems to me it breaks a key premise of peer review (even single-blind). Namely, now you have implicit and explicit pressure among reviewers.
What are the arguments for this? ("This" being: divulging the identities of the reviewers to the other reviewers of the same paper -- not only to the AC)