11

I have often seen people(postdoc, etc.) listing in their CV or webpage that they were awarded best reviewer for conference X. I have recently been invited to become a reviewer for a conference, which has made me ponder over what are the criteria for giving such awards? (My field is computer science if that makes a difference.)

  • 2
    I've never heard of a "best reviewer" award so this certainly isn't ubiquitous in computer science. But what makes you think there's any criterion other than, you know, being the best reviewer, in terms of thoroughness and helpfulness? – David Richerby Jul 26 '14 at 20:41
  • I am really curious about this: in my field (mathematics), we do not have a separate reviewing system for conference papers (most conferences talks are not directly linked to papers at all; in the ones which are, the papers come after the conference and, if they are refereed, are refereed in the usual way, not by some pre-assigned group of "reviewers"). Is there some small group of reviewers who are all reviewing the same papers (and are qualified to do so? impressive!), so that it makes sense to identify one as "best"? Very interesting. – Pete L. Clark Jul 26 '14 at 20:42
  • @PeteL.Clark You should probably ask a question about how that works since it's too much to explain in a comment. Executive summary: it's organizationally similar to journal refereeing, in that the papers submitted to a journal in any one year will have been reviewed by many different people and ditto the papers submitted to a conference. But it all happens within a month or so, which means that the reviewing is rather superficial. – David Richerby Jul 26 '14 at 22:41
  • 1
    @PeteL.Clark No more unfair than designating one paper "best", given that everyone was writing different stuff? But, as I say, I've never come across a "best reviewer" award so I'm winging it only slightly less than you are, here. – David Richerby Jul 27 '14 at 0:20
  • 1
    BPM (bpm-conference.org/BpmConference) has a best reviewer award AFAIK. It is a simple vote by paper authors (i.e., every author is asked for feedback on the value of Review 1, 2, 3). I am not sure how far this is away from entirely random, as I presume that authors of rejected papers are typically not satisfied with their reviews, but authors of accepted papers are. Anyway, a friend of mine got this award two years ago, and he does write very thorough conference reviews. – xLeitix Jul 27 '14 at 8:10
4

Here are the criteria used by the ACM 2012 International Conference on Multimedia Retrieval (ICMR) for the "best reviewer" award:

The criteria are that the reviewer should have demonstrated the expertise to provide insightful and knowledgeable feedback, should have provided quality reviews which are helpful to the authors of papers, should have displayed collegiality and constructive feedback, and should have completed reviews on time. The award is selected after examining all of the 400+ reviews from the 101 reviewers.

Conferences with an author response phase may use the authors' replies to help choose the best reviewer. For example, the CFP for the 2014 Conference on Software Language Engineering says:

Authors will be given a brief opportunity for a reply to the reviews. The replies will be considered in the PC's discussions, and considered in the selection of the best reviewer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.