Is there a "Rate my Advisor" type website?

Question

I am not sure about the incentive effects of websites such as ratemyprof.com, as Professors with tenure may or may not care about undergrad students. However, they may be more serious about graduate students (supervision, co-authorship etc).

Does a ratemyadvisor type website exist?

This is important (I think) because relationships between supervisor-supervisee are probably more profound (given the time investment) and have arguably more ramifications on the future (reference letters bear more weight).

Any thoughts are welcome!

Answer 1

I spoke recently with someone who is building such a website/community, and I doubt they will mind me giving a brief overview of how they intend it to work.

Basically there are 2 issues that are difficult to overcome - assuming all the technological issues are minor:

1. Skepticism
2. Slander

Skepticism in the sense of 'Oh, I see this PI has a bad review from 2001, but who's to say that will happen to me. Maybe this one student is an odd-ball'. It also has parallels with the TornadoGuard problem.

And we see this a lot on academia.stackexchange too. We only ever hear 1 side of the story, and that story is rarely impartial. You know, occasionally people like to rant. People often omit the times they made mistakes. It's human nature. And so any website of this nature has to somehow overcome this issue of he-said-she-said, and theres simply no technological device we can implement to get around this. Particularly if you allow anonymous submissions.

The second, slander, is what I believe most people in the pubpeer.com debate are worried about. A deliberate attempt to hurt someone's reputation under the flag of mob-justice. Nitpicking and emotive words, but no real substance, etc. I actually only found out about pubpeer a week or so ago when a colleague at a party showed it to me, and I was shocked to see people I respect being picked apart on there for silly little things. Still, those anonymous criticisms do have some validity, and the concept of post-publication peer review is something I deeply agree with. I think only time will tell what impact sites like pubpeer will have on science. For a PI-review site, it could have disastrous consequences. Potentially legal consequences.

The website/community I referred to up top seemed to have spent a considerable amount of time fleshing out all the technological methods that could help here, and in the end i'm told they settled on the fairly low-tech idea for Skype-based interviews, potentially on a weekly release schedule. The idea is to mainly cater for people who hate their PhD and feel relief knowing they're not the only ones going through hell. So these Skype interviews generally are with post-docs or no-docs who had a bad experience, and they talk about what the end result was, what they'd do differently, if they still feel they were treated unfairly many years down the line (I watched 3 interviews and they were all totally different, so i'm finding it difficult to generalise here). But the big issue for the team putting all this together is finding PhD/post-docs/no-docs who are OK with talking about their experiences. They offered to interview me but I turned them down because, frankly, i'm not that brave. They offered to pixilate my face, but even so... as disappointed as I am with my treatment during my PhD, there's no one specific person that really let me down. It was a system of failures, I think, and one that is unlikely to happen again. So that particular PI-review project wouldn't really help me or people like me.

Answer 2

To add to the issues mentioned in Wetlab Walter's answer: one of the common reasons why advisors and student relationships break down is a mismatch between their personalities. It causes unrealistic and often unsatisfied mutual expectations.

Therefore one's perfect advisor can be another person's nightmare, and vice versa. Even a good track record with several past students does not reliably predict unqualified success with the next one - partially because typically the sample size is very small, but mostly, I think, because of the high degree of subjectivity about what a "good advisor" really means.

Answer 3

Such a website will never work. Most professors have fewer PhD students in their entire career than they have undergraduate students in a single semester. They will never accumulate enough reviews to be useful.

However, you can get the information. Simply email the professor's former students asking them for advice. Often a list of these students is on the professor's website. If it is not, ask the professor for the list.

Answer 4

Yes, there are such websites.

I know of the following two:

(i) http://www.qcist.com; (ii) https://www.ratemypi.com

However, at least at this moment, these websites remain useless and mostly abandoned. Note (i) is free and (ii) is paid. This is because to, in addition to a number of issues related by other respondents, academics are remarkably overreactive to criticism. This works directly against open criticism, as public relations play a central role in defining academics careers, and petty individuals are frequently empowered enough to take revenge on others. I am sure a number of PIs would exploit (public) institutional resources just to try and tear such websites down should they take root.

Answer 5

Yes, there are at least two websites.

1. Professor Anuj Sharma from Iowa State University developed one: https://ratemygradadvisor.com

2. Two PhD students developed one: https://pi-review.com/

The first one primarily requests ratings from one star to five stars, accompanied by a brief comment. The second one asks for more detailed comments in response to specific questions.

Answer 6

I strongly disagree with the opinions here. If you're worried about people slander-ing their PIs, then implement one or more of the following ideas:

1. Allow them only a Likert scale of predefined queries about their relationship with their PI (i.e., no free response answers).

2. You can make them identify the top and worse qualities of the PI, and if they fail to select top qualities, not let them submit the form.

3. You can make the reviewer review more than just 1 PI to get a sense of their attitude/outlook tendencies. You can flag students who frequently underscore their collaborators so that viewers making judgments about prospective collaborators will contextualize the feedback.

4. You can make the reviewer specify how many hours or years they spent working with the PI being rated, which adds further context, and to label their relationship with them (advisor-advisee, collaborator, committee member, in my department, talked to them once, interviewed with them, etc).

5. Allow reviewers to report the theme of the abuse, but not free response details. Categories can include criminal-oriented offenses (sexual misconduct, physical harm), moral offenses (verbal abuse/yelling), personality disorders (narcissism, borderline), or something more subjective like "doesn't trust others", "doesn't show respect", "is controlling", "is a bad listener", etc.

6. Allow an optional identity verification for past students who don't care if their advisor knows it's them who reviewed them. For PhDs who leave the field after they graduate, I doubt they'd care if their advisor knew they rated them poorly.

It is sooo easy to implement a fair enough reporting system, and I resent the opinions that say otherwise. I feel you are just supporting the hazing, abusive nature of many advisors in academia. In fact, bosses should not be afforded more protection than their underlings (including tenure, imho, but whatever, it's not my money to disburse). If a PI behaves badly, they behaved badly. I don't CARE if the student made a mistake or was negligent, and that this elicited the PI's behavior. Mistakes do not warrant abusive behavior from a boss. Negligence could be a flight response to the PI's abusive nature, and fight or flight is a survival instinct. People in positions of power must be held to higher standards, and instead, we are protecting them.

A web app that allows students to give feedback on their PIs'/collaborators' leadership qualities will not fundamentally determine hiring/firing decisions - but hiring managers should be given more accountability/motivation to do something with student feedback. As a grad student, I did an exit questionnaire where I exposed my PI's abuse, and nothing was ever done. I suspect it was because this feedback was not made public. In fact, I don't think anyone but my graduation officer read it, and he has no motivation to do anything about it, nor the power to do anything about it. Did my PI commit a crime? No.. but abuse is a moral crime, and people should be warned against giving bad PIs more power, more money, and prospective employees should RUN FOR THEIR LIVES. We deserve that kind of transparency.

And it's simplistic to tell someone to ask current graduate students about their PIs, and ironic because one of the first criticisms for a web app feedback system in this thread was that it was likely not anonymous due to the small population size of possible reviewers. If a first year asked me about my PI prior to me graduating, I could not tell them the full truth because them changing their interest in joining the lab would have directly implicated me. You can't criticize an online portal for the same deficiencies your alleged solutions possess and call it a better system.

Answer 7

As I knew such website doesn't exist yet. It would be a nice idea for creating an app for this work:) Anyway about getting to know the professors in a given department is not a difficult task and you can do the following,

1. See the website of a given advisor to learn about recent publications, grants, and number of current students

2. If you can visit the depatment, go to it and speak with the current students and also his(her) students taken a course with the advisor

3. If you cannot go to the department, find the email addresses of the current students of that advisor and ask important questions about its course works, research process, active topics in their lab

4. If you are decided to work with a advisor based on your findings and interest to his works, contact with him and just ask some formal questions to see its rules on admissions (if a busy supervisor reply your email, you are luckyman!)