I have often seen people(postdoc, etc.) listing in their CV or webpage that they were awarded best reviewer for conference X. I have recently been invited to become a reviewer for a conference, which has made me ponder over what are the criteria for giving such awards? (My field is computer science if that makes a difference.)
Here are the criteria used by the ACM 2012 International Conference on Multimedia Retrieval (ICMR) for the "best reviewer" award:
The criteria are that the reviewer should have demonstrated the expertise to provide insightful and knowledgeable feedback, should have provided quality reviews which are helpful to the authors of papers, should have displayed collegiality and constructive feedback, and should have completed reviews on time. The award is selected after examining all of the 400+ reviews from the 101 reviewers.
Conferences with an author response phase may use the authors' replies to help choose the best reviewer. For example, the CFP for the 2014 Conference on Software Language Engineering says:
Authors will be given a brief opportunity for a reply to the reviews. The replies will be considered in the PC's discussions, and considered in the selection of the best reviewer.