I have rewritten this a few times trying to find the right approach to this; forgive me if something ends up mangled in the process (and the length, of course). As non-faculty holder of a recent terminal degree, I wanted to weigh in from the student side of things.
- there are a lot of intersections between things we perceive to be part of a person's "personality" (charismatic, nice, frank, honest, mean, personable, distant, etc.) and their ability to work effectively with students
- it's a big concern to me if a hiring committee is picking someone they'd like to have a drink or a dinner-party with over someone who would make, you know, a good member of the faculty; I want to have faith that these decisions are being made with the theoretical student's best-interests in mind, because we the theoretical students may need you to have our backs.
I'll do this backwards and start with the second point: as students, we don't have a lot of leverage regarding who is and isn't faculty, and if serious concerns with another faculty membercome up, it's important that, as students, we know these concerns won't be dismissed because everyone pals around with them and can't see their flaws. This is a balance, of course; I enjoy faculty who get along, and I've learned a lot by being mentored by more than one faculty who are good friends.
To the first point: While I was a graduate student we were asked to attend talks by the three candidates given on-campus interviews and give feedback on the candidates. We debated the tradeoffs between which candidate we felt produced the best work and which candidate did the best job of actually talking about that work in a way that (we hoped) would make them better in the classroom and as mentors.
I recused myself when car trouble popped up and kept me from attending the third talk, but of the two I did see, the more charismatic candidate won the position over a candidate who appeared more insightful, kind and levelheaded but had trouble communicating the excitement and importance of his own work. This was a problem when going up against someone well-known in his field. How much faith can I put in a professor who is uninteresting even with the agency to choose topics and incentive to shine (compare to: when he is teaching something he doesn't want to teach to students he doesn't think he has time for)? At the time, this decision satisfied me, though there were certainly others in the program who thought the decision was terrible.
In retrospect, the candidate hired is a charismatic person who is very successful in his field and isn't afraid to give feedback that we as students may need to hear, but which most professors are too nice to give. He can also be a boor, sexist, bully, and drunk. He clearly played favorites with attractive female students, gave unattractive female students a hard time, while being much more even-handed with the male students (with a notable exception regarding a student interested in the same female student he was...). I won't be surprised if I hear certain kinds of news about him in the future.
I've had nice, friendly charismatic professors who are good at guiding students, mediating conflict, treating everyone fairly and encouraging us to succeed. I've had harsh but charismatic professors who never run out of brutal honesty drive us all the harder to succeed. I never had a professor wow me into being a better student by the weight of their accomplishments. I've also discussed a charismatic professor whose extracurriculars were a distraction to our ability to learn what he's capable of teaching. None of these factors will generally stand out on a CV, and some of them will obviously prove elusive even after a thorough campus interview.
From my distance, I can only hope our faculty and administrators weren't picking a friend, and will be capable of sober reflection and thoughtful advocacy for student interests should a student come to them in need.