This is a second attempt to ask about peer evaluation, after a pretty unsuccessful one which was probably much too broad. Let me start again, with a more precise subquestion.
The point is commonly made that one should not judge one's peer by looking at the impact factor of journals where her papers are published, as IF is a very poor indicator of the citation rate of individual articles, and certainly even poorer at asserting the intrinsic merit of individual articles. Some go further and consider that one should not judge one's peer by looking at the prestige of journals her papers are published in (e.g. because more selective journals tend to publish less reliable science). Last, the sheer number of publications is obviously no more useful to judge one's peer than the sheer number of books written is useful to judge a writer's merit. So, following these principles, one is lead not to use publication lists in peer evaluation.
However, publication list is most certainly the main component of a record in many, if not most, evaluation.
I have several questions on this paradox, to which I have personal answers which are partial and non definitive.
Q0 What are the actual practices you witness about the use of publication lists in peer evaluation?
Q1 Are there circumstances where using IF, journal prestige or other aspect of publication list leads to a better evaluation than not?
Q2 What other proxy if any can be used for evaluation that are expected to be done rather quickly (e.g. extracting a short list from dozens to hundreds of applicants for a tenured position, attributing small fundings), and what are their pros and cons?
Q3 What criteria can be used for evaluation that are expected to be done more thoroughly (e.g. hiring for a tenured position once a short list has been established), and what are their pros and cons?