20

Every year we organize a competitive international call for PhD students (in the area of biology). What measurable criteria should we use to predict their academic success and award them research fellowships?

I realize that part of the question is ill-defined because it is not clear how to define “success” for a PhD student. But since I imagine that many of us have this problem and have potentially thought of a solution, I would love to read your thoughts on this question.

| improve this question | | | | |
  • 1
    As a graduate student, I would say that important qualities include persistence, creativity, and of course hard work. It goes without saying that one should be able to learn well. – abnry Mar 2 '14 at 18:56
  • 1
    So you are looking for answers which will first define success and then show which indicators best predict that success for PhD students? – earthling Mar 3 '14 at 4:56
  • Ideally yes. But an answer addressing only one of those would be interesting. – gui11aume Mar 3 '14 at 8:59
  • 1
    Maybe this might help provide some insight? It's only on the course level but it would be a place to start. ncbi.nlm.nih.gov/pmc/articles/PMC3002135 – Irwin Mar 3 '14 at 21:11
11
+100

The major quantifiable predictor of success in research is... success in research. People who have done research successfully in the past are more likely than not to continue to do so.

For students that have not done research in the past, the best predictor I have seen for success (whatever that may be) is expressed in a quote from The Unwritten Rules of PhD Research, by Marian Petre & Gordon Rugg:

A willingness to learn for themselves and good judgement about when to stop and ask for feedback.*

It's not exactly quantifiable, but you can get a sense for it in an interview. (Of course, this quality can be learned, so a lack of it doesn't necessarily predict an inability to succeed at research.)

* I took this quote completely out of context; the authors there are actually discussing the role of the PhD advisor, and they mention this quality in reference to a student "who can be pretty much left to get on with it, with supervisory meetings being something that both parties enjoy, and where each party learns from the other."

| improve this answer | | | | |
  • 2
    This is absolutely right. The only really telling metric is previous publication success. The difficulty of course is that not many people are going to have good publications before they've even started a ph.d. program. So this is the right answer, but probably not all that useful given the population the OP is trying to evaluate. – shane Mar 4 '14 at 21:02
1

Short answer: You should not be looking at what people do or how they are, but how what they do looks like.

This is, how are they qualifications, specially considering consistency between them, it's not about the average, but the average and a small standard deviation. The greater the deviation, the less predictable they will be.

On a side note: you will probably reject geniuses.

Long answer:

"Success" is now defined as "publishing", it's pretty clear how to define it since people that have to define quality and metrics are focusing on this.

This may be wrong, some people say it's wrong to use P-values to test your hypotheses, some people say it's wrong to use h-indexes to measure the quality of researchers, but it's certainly becoming more and more common. These values provide a warm feeling of objectiveness and it's very hard to refuse to that. IMHO, they are here to stay, and without any doubt, they are here.

Having clarified that, I have a personal hypothesis (not verified at all, sorry) that people that get good grades are better at publishing. The reason is that we can consider that the corrections of an exam and the reviews of a paper are similar.

In my experience, in both cases is not about how much you know or how much you can do, but on the contrary it is about:

  • Conformity. Using the same language, terminology and not producing something shocking or hard to understand that will cause failing the exam or the paper being rejected.
  • Writing skills. Trying to predict (consciously or not) how the person reading your paper/exam is going to interpret it, avoid misinterpretations, show self-confidence, clear ideas, clear structure, etc.
  • Concision. This is more than a writing skill. Time is limited in exams, pages are usually limited in papers (and the less pages, the more papers, that's also good, in principle). But it's even more than that, because most of the time it's not about how much you know or how much you have done, but avoiding mistakes. An exam that replies perfectly to half of the questions (and only that) will look better than one that replies perfectly to 90% of them but then makes really stupid mistakes in the other 10%. A paper with a small contribution may get accepted (depending on how small it truly is), but a paper with an important contribution and then an important mistake will get rejected (even if the mistake is only in the mind of the reviewer because the terminology used does not conform to what is usual and this makes reviewers very confused).

So it's really about being compliant with the state of the art and moving step by step further, with small steps, baby steps, avoiding mistakes. How can you know whether someone can do this? Looking at their grades, and specially to the deviation of their grades from the average, it's not just about high grades, but consistent grades. It's not about how much they know, but about how often do they mess up, because if they do often, chances are they will do it at least once per paper, getting all rejected.

This is the case for me, from time to time I'd do something really "brilliant" in an exam, the teacher would not understand it and I would get a qualification of 0 in that exercise. I have never cared about qualifications, but learning, now publishing is just like any other qualification. Success is just like any other qualification.

| improve this answer | | | | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.