I have been on two such panels, both for departments in my field,
one at a 'sister' university within the same system as my own, and
another at an unrelated institution in another state.
My first observation is that you need to understand who will receive,
evaluate, and perhaps act upon findings and recommendations of your
panel, and what criteria they will use. If you are not personally in substantial
agreement with stated institutional goals, you should probably decline the
opportunity to serve on the panel. You might be able to 'score a few
points' for an alternative institutional policy in the panel report, but typically
these reports are not made public and it is unlikely a lone voice will have
much effect changing goals. Also, there may be an element of unfairness
in judging a program by goals at odds with stated institutional goals.
Before your visit you will probably get a huge packet of information about
faculty achievements, student and researcher outcomes, enrollment and employment trends, and budgets.
Beware that the most important information may not be highlighted, and
look for obvious gaps in this information. Omissions and inconsistencies
can be a guide as to what questions to ask when you get to campus. Of course,
the institute, school, or department under review has a right to present
its best case, but you will do your best job if you look beyond the PR gloss.
Focus on what is actually going on, as well as historical information and
stated goals for the future. It is easy for a faculty member or researcher to claim
a passionate interest in various trendy fields, but you should look at what she/he
has actually published in the past few years and evidence of current activity.
It is easy for a department to claim increased student and staff diversity as a goal,
but you should look around to see what is actually happening. You should
have as many one-on-one meetings with faculty, staff, and students as possible.
Ideally, some of these meetings will not be pre-arranged. On your own,
have a slice of pizza at a local hangout; sometimes that can give you more
information (or clues to search for information) than prepared binders of documents and fancy receptions with carefully chosen people.