6

Is there any US grad school ranking based on graduate students' satisfaction? (satisfaction while in grad school of after graduation)

If department-specific I am mostly interested in computer science departments.

5
  • 6
    There might be a big difference between "being happy" while in grad school, and "becoming happy" subsequently, and with some hindsight and maturity. Jun 4, 2015 at 20:20
  • 3
    I generally take measures like "graduate students' satisfaction" with great caution. On the one hand, it is a very subjective measure, which in itself is not a problem, but which becomes problematic because few, if any, graduate students can compare between two, let alone more institutions, while at the same time being strongly influenced by people at their own institution. On the other hand, it is a very wide measure, which in turn can lead to students cling to arbitrary, but verifiable criteria, to judge in a reproducable way (that, however, may be rather disconnected from their ... Jun 4, 2015 at 21:49
  • 1
    ... actual satisfaction). Jun 4, 2015 at 21:49
  • 1
    I think this is an excellent question (although I anticipate the answer is "no"). While graduate students' happiness with their institution is certainly a subtle issue, it is definitely a problem if the majority of a department's students wish they were not in that department! And gauging student satisfaction is one of the main reasons to visit a prospective graduate school. (If you visit a department and the current students advise you to run away, you should probably listen to them.)
    – Jeff
    Jun 5, 2015 at 0:18
  • 2
    Furthermore, I have some rather satisfied friends doing PhDs that seem to suffer from Stockholm syndrome. "My supervisor is very nice, I only have to sacrifice my first born to her".
    – Davidmh
    Jun 5, 2015 at 16:20

1 Answer 1

5

You are likely to find there is no such ranking, and if it existed you should probably throw it away without looking at the results - for a rather simple reason! There exists no standard assessment of graduate student satisfaction that is widely applied (I think not even in relatively small regional areas, counties, states, etc), and there is a good chance it could vary wildly according to what time of year you applied it to students. Also, existing published results are not actual research and are biased advertising pretending to be serious academic inquiry.

Published Satisfaction Results Are Extremely Biased

Whenever you see analysis of satisfaction results and such, note how they are remarkably positive slanted? That's because this isn't so much research as it is advertising. I'm going to get a little soap-boxy now, because this bothers me. Take for example the MIT survey and it's results. Note the questions like this:

Overall, how would you rate the quality of...

Your academic experience at MIT? Poor, Fair, Good, Very Good, Excellent

Do you notice something funny with that scale? Of 5 choices, 3 are positive, one is neutral (fair), and one is negative. That perks my ears up, because a normal bi-polar Likert-style scale like this is "Very Poor, Poor, Neutral [Average/Undecided/etc], Good, Very Good". Why would they choose an unbalanced, biased scale like this which would be utterly unacceptable in any legitimate field of psychological assessment and research? The item, as posed, is designed to elicit positive responses.

Further, the vast majority of items are asked positively as in "how satisfied" compared to "how dissatisfied" - in general serious psychological assessments should ask in both positive and negative ways to attempt to remove a positive-selection suggestion for the measure as a whole. That is, if the measure is designed to get at how people really think and feel, rather than being an exercise in "tell us how great we are so we can tell people how great you said we are".

If this were peer-reviewed research to be published in a reputable journal, it should be rejected. I suspect all institutions run similar games, so I don't mean to pick on any one institution in particular - but I hope this helps to illustrate how little stock you should put in such reports. It's not science, it's not even good polling - you are reading an advertisement from the institution designed to make you want to buy the product they are selling.

Interpreting People Is Hard

As a matter of psychological assessment, how the question is worded and it's context and framing within the measure itself can strictly dominate how the question is answered, thus the need to compare tests that have been proven comparable. Due to advances in our understanding of people's ability to judge their own satisfaction or happiness, we now know also that how such a question is asked will change how a person evaluates the question. Asking someone "Overall, how satisfied are you with..." will tend to elicit answers from the autobiographical "remembering self", which is based upon a reliably biased self-conscious memory system. On the other hand asking "Lately (or over the last few months), how satisfied are you with..." - or even just generally asking about feelings, emotions, or 'happiness' - will actually elicit answers from the "experiencing self", and will instead be biased based upon things like emotional state and very recent experiences and sensations.

If the questions are worded to call upon the remembering self, with questions like "overall...", based upon the literature I would expect institutions with economically affluent student bodies to show much higher satisfaction ratings than institutions with a less wealthy student base - even if the less wealthy institution were actually a much better place or at least no worse. Wealthy is positively correlated with this kind of satisfaction, and there is no limit - the more wealth, the greater the satisfaction overall. So a measure of graduate student satisfaction would actually be a proxy test for "where the rich kids go".

On the other hand if the question calls upon the experiencing self, I would expect wealth to have a much smaller correlation generally with the largest effect being seen by students living below the poverty line. So campuses who ensure - through admissions or financial aid - that students don't live below the poverty line would do better than others, regardless of academic quality generally.

Daniel Kahneman gave a nice TED talk about this subject, if you are interested in a nice overview of the topic.

The MIT results are a good example of this - the MIT results show that 88% of students were at least somewhat satisfied with the graduate school, while 81% were at least satisfied with life outside the institution. In other words, people rarely say "gee my life sure does suck in general, but MIT is great!" The two questions are likely to be so strongly correlated that they are actually asking the same ultimate question and respondents may be literally incapable of attributing their general satisfaction to any specific source. "When you're smiling, the whole world smiles with you."

What's worse, this problem would extend to both currently enrolled students and follow ups after graduate school. A useful study of such rating would have to get more than just the answer to satisfaction, and would have to also collect other student information and attempt to control for other known factors (marriage, having children and what age those children are, health, etc) before we could even usefully interpret what a statement like "88% of students say they are satisfied with us" means - is that good, or terrible, or meaningless? We just don't know.

MIT's results alone help to point out how odd the responses are, when you think about them - at least, they are odd if you think that people's answer to such questions would be overtly sensible and comparable. 78% said their academics were "very good or excellent" (not the same scale as satisfaction, note), 56% said the same of student life, yet 88% claimed to be satisfied with MIT - so 1 in 5 say "academics are blah", and nearly half say "student life is ehhh", but 88% say "yeah MIT is great". Well, that just seems odd. And 81% say they they are satisfied with their personal life, yet 35% were not satisfied with their ability to combine the needs of academic and family life - life inside and outside the institution is asked to be a reflection of the institution, but combining both is requesting the respondent to criticize themselves. That's an interesting reflection on the measure itself, and makes the results even harder to interpret.


Some Interesting Actual Research

Graduate student satisfaction is not without some serious research, however. The largest attempt at seriously talking with graduate students and getting their opinions and feelings seems to be the PhD Survey. It managed to get responses from other 4000 students from some 27+ institutions, with a rather high response rate (over 40% at least) on a 20 page survey. It probably has the capacity to be used to make some comparisons between institutions, potentially, but the N of such comparisons will drop remarkably and would require access to the underlying data - and appears to not be published.

The study is so interesting though that I think just trying to rank institutions would be a sad use of such interesting insights, and even at least making comparisons between institutions would still be hard for the previously mentioned readings.

That caveat aside, I highly recommend the reports from the linked site - they make a fascinating read for anyone considering - or enrolled in - a PhD program.

Bottom line (TLDR;)

Even if people were honestly reporting unbiased findings - and they generally aren't - and people's responses could be interpreted in a clear and sensible fashion - and they can't - you wouldn't be able to interpret the results in a useful way or be able to compare different institutions.

A 3rd party poll with peer-reviewed published results using an appropriate measure would be potentially really interesting - but sadly I don't think that exists today. But if it did I would love to read it!

Some interesting research does exist about graduate student satisfaction and program effectiveness, but I have not found any that attempts to rank institutions, and I would propose it might very well be because we don't know what to rank them on! As shown in the PhD Survey, graduate programs tend to be focused on preparing participants for jobs they are unlikely to hold and not preparing them for jobs they will have - yet the unemployment rate is low and other studies show clearly that average income is still well above bachelor degree holders generally.

The studies that are done also highlight the fact that each department - or at least major degree field - has some very different experiences. While one field might have 17% of graduates working in tenure track jobs, for instance, another has nearly half of graduates in such jobs; in some fields a minority (30-something percent) of participants are interested in tenure-track faculty jobs, while in others more than 80% are so interested.

As such it would be potentially possible to compare individual departments or degree fields across institutions, but making such comparisons among institutions would likely result in highly deceiving results that one should not attempt to interpret naively.

3
  • 1
    Thanks I'm indeed ideally looking for a 3rd party poll with peer-reviewed published results using an appropriate measure. I'm fully aware that the MIT example I pointed to is mostly a publicity stunt. Jun 5, 2015 at 18:17
  • 1
    @FranckDernoncourt I'm glad :) With that said, I have found some research that might interest you, though so far I've found nothing that considers ranking Universities.
    – BrianH
    Jun 5, 2015 at 18:56
  • 1
    I have updated my answer with some interesting research on PhD program surveys (next to last section).
    – BrianH
    Jun 5, 2015 at 19:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .