I may have to write a bad recommendation for an underperforming student researcher in the Fall. How to avoid this without being exploitative?

Question

TLDR: PhD student mentoring an undergrad in super competitive field. Undergrad has an amazing non-research CV, but no research experience. He says he's motivated but got very little done in 8 months. I will have to write him a letter of recommendation in the Fall and if things continue, it won't be good. What should I do now to avoid this without pressuring him too much into doing work?

I'm an international PhD student doing Machine Learning at a top-4 US university. Last Summer I started mentoring 3 great students from my home university trying to create a win-win: I teach them about research and ML and help them get a paper to have a chance at entering a good grad school (ML is so hot it's now super hard), and I get help from top, motivated students and improve my mentoring skills. Before that, I had already mentored 10 students from the US university with the full range of success: from students that stopped coming after a couple of months, and 3 others that published with me, applied to grad school, and got into another top-4.

The plan with these 3 students was to work virtually (everything is virtual in 2020-21) and I would try to bring them as visiting students once we could show something to my PIs. Out of the 3 students, one is doing amazing work and we invited him to come this summer and another realized he didn't like research and quickly dropped out. In a way, both are good endings.

The third student is essentially the perfect undergrad: won gold medals in multiple national olympiads, top 1-3 in his promotion while doing 2 separate degrees in Math and CS, and winning international CS competitions in undergrad. However, you now need research to get into a top university in ML.

• During the Summer, the student and I were meeting 5x per week and he did a bit of progress, but much less than I expected and a bit below the average of the students I've mentored. In particular, I gave him a concrete project which I was hoping to submit on a September deadline and it's now February. I avoided telling him anything because he's doing it for free: no credits, no money, just for the CV. He didn't realize he hadn't performed well during the Summer until I told him much later.
• In the Fall he got close to nothing done even though we were meeting 3 times a week. In November I told him: "Hey, we're not making progress, maybe we should stop working together. We're losing our time and I would like the project to be completed since ML moves very fast.". He told me he's very motivated and he'd put more hours in. Didn't happen.
• In January the subdepartment of my home university where I studied contacted me: I had previously convinced my PIs to bring one student to do a final degree project. This subdepartment has gained an international reputation and now has lots of spots all over top places for students to go. They do a matching so that students with the highest GPA get to choose where they go first. Because the non-working student has a top GPA he got to choose me. So next Fall he's coming to do paid full-time research with me. This also "guarantees" that one of my PIs will write him a letter based on my first draft.
• In February, he asked me for a letter for a fellowship. I told him I couldn't write him a very good letter. He said he hadn't really understood that getting into grad school implied having good research, not just good GPA and Scientific extra-curriculars, where he was indeed investing a lot of time. So we agreed I would write that he had lots of potential, but he hadn't had much time because of all his academic commitments. This was indeed 100% true and maaaybe good enough for the fellowship, but it's not going to cut it for grad school. He told me he realized that he hadn't done much and he would now devote 20h/week. Spoiler alert: hasn't happened. In the meantime, the idea I gave him is stuck, terrible in a rapidly moving field, and it's increasingly likely he won't have anything published by the grad application deadline.

Having now mentored 15 students, I'm pretty confident I'm giving him reasonably easy work. I've kept simplifying and concretizing and for the last few months, it involved just something slightly more complicated than the introductory 1h Deep Learning tutorial. I once tried myself to make progress on what I was asking him and got more done in 2h than him in 2 months.

I believe he thinks that because the subdepartment has many students in top places, and he's one of the best in GPA, that he'll get accepted. But, having helped with ML admissions, he doesn't stand a chance with zero papers. More importantly, right now my opinion of him isn't good and a sincere letter (which I 100% intend to write) would have to convey this. However, I've read in this community that you shouldn't write a negative letter.

Given that there are 8 months left, some of which he'll be working full time, there is time for a change. What can I do to maximize the chances of a good outcome? Some solutions involve being extremely pushy. However, I have seen what exploitative advisors can do to some friends of mine and I don't want to be such a PI. Finally, not sure if relevant context, but I'll be applying for professor myself next Fall.

Answer 1

Your job as a mentor is not to “maximize the chances of a good outcome”. The problem with that mindset is that your definition of a good outcome won’t be the same as everyone else’s. And an outcome that might appear good in the short term (e.g., getting into an ultra-competitive grad program) might end up disastrous in the longer term if it is achieved by someone who doesn’t have what it takes to succeed in such a program.

So I suggest adopting the mindset that your job as a mentor is simply to offer good mentorship. Give good advice, give honest, accurate information about what it takes to succeed in research and to get into a good grad school, and what your standards are for writing a helpful letter of recommendation (and of course, as you yourself said, don’t compromise your integrity when writing the letter). However “good” or “not good” the outcome turns out to be, if you did those things then you’ve fulfilled your responsibilities as a mentor in the best possible way.

Answer 2

right now I'm very dissatisfied with him and a sincere letter (which I 100% intend to write) would have to convey this

First, let's cut the student some slack. They are probably exhausted and burnt out from all the coursework and other academic activities. The problem doesn't seem to be laziness; rather, it seems to be self-awareness: they overestimate the number of things they can do well at any given time and underestimate the importance of having successfully completed a (research) project.

The dilemma you must face is how much of this to discuss with the student.

• On one hand, it is worth making your expectations (and grad schools' expectations) clear; you don't want them to say "I did everything you told me and got screwed" when grad school decisions come out. And certainly, there is a limit to how much time you want to spend on a student that is not performing.
• On the other hand, forcing students to be successful undergraduate researchers is not necessarily the right thing (even if it were possible); students who have unsuccessful research experiences and end up with mediocre letters of recommendation and no publications can learn a lot in the process (both about science and about themselves).

So next Fall he's coming to do paid full-time research with me...What can I do to maximize the chances of a good outcome...[without] being extremely pushy?

I have two pieces of advice. The first is to make your technical expectations clear. Presumably, you want this idea submitted to a journal by November. You can work with the student to come up with a roadmap for this, with interim milestones. For a student, seeing the long-term plan, with a concrete outcome, can be very helpful. And both of you can objectively measure the student's performance against the roadmap; there are no surprises in the fall.

The second is an attendance requirement. There is no need to change your supervision practice, but consider requiring the student to spend time physically in the building doing work each day. You may consider this pushy, but for paid, full-time research, I think it is reasonable, and has two key advantages:

• Sometimes getting started on research is the hardest part. After things start happening, the project becomes fun.
• Lots of great interactions happen between people who are physically writing code together. Certainly the student will learn more from you if you can drop by their desk and work together at the same computer. And if multiple students are working in a room together (even on different projects), they can learn from each other as well.

I'm not suggesting that the student has to do the entire project at their desk in the building, but requiring their physical presence, say, 6 hours a day for the first month or two might help break the logjam and get the project moving. And certainly it will give you a better view of the situation, which can only help your letter.

This also guarantees that one of my PIs will write him a letter based on my first draft.

I would consider discussing the situation with the PIs now; this brings the potential issue to their attention early, and they may be able to provide advice that takes all the individual local factors into account.

Answer 3

If you think your letter won't be good, you should refuse to write the letter.

Answer 4

It is obvious that the student is highly capable, but simply has too much going on in their life, and haven't had time to dedicate themselves to this project that they are doing with you for free. So obviously your work will be at the bottom of their priority list.

To me, it seems obvious what's going on. The student signed up for this project so he could write on his resume "I did this". You signed the student up expecting him to deliver tangible results. These are two different things entirely. The student is just looking for experience, you're looking for results.

And honestly, I think that's on you. Considering the way you've sold this idea to your students, and considering the way it sounds like your student is very confused as to what you expect from him, it sounds to me like there's been a complete miscommunication (from you) regarding what this project is actually about.

If you wanted him to produce results in a summer, that should've been made clear from the get-go. But the fact that you expected results in September and yet you didn't even tell him that until February, is a clear sign that you might be the one mishandling this situation, and causing confusion due to bad communication.

Answer 5

It is not your job to get the student into graduate school. As a recommender, it is your job to decide if you want to recommend the student and to write an honest letter if you do. You may wish to give the student advice on how to improve their chances of getting into school, but ultimately it is their path.

There is nothing forcing you to write a letter. If you do not wish to do so, then you should politely decline.

If you do decide to recommend the student, your letter should focus on the positive qualities that you can speak to. It would be good to communicate to the student how much stronger your letter would be if they had more research experience. But your letter should focus on the student’s accomplishments and potential that you see. As another answer has already pointed out, the student might be exhausted with all of the work that they have done; perhaps you could discuss that in your letter.

Ultimately you do not decide if the student is fit for graduate school. The admission committee will see the lack of research experience without you pointing it out in a letter. Again, it would be good to communicate frankly with the student about how much stronger their chances would be if they improved their research, but they are entitled to their own path.

Answer 6

If your account is accurate the this student is clearly prioritizing his undergrad studies over this research assisting work. And it also shows that you have been too willing to entertain promises of future work from him than you have been to insist that he call it a day.

Your post highlights so many things wrong with academic research today.

Having bright overseas undergrads sent to top US university departments for a prelude to "research" on a topic chosen for them rather than something they envisioned themselves and having them "supervised" by PhD candidates just a few years older than them is just plain foolish.

They would be far better focussing on their own final year undergrad dissertation (on a topic they devised for themselves) with constructive comment from a lecturer. That is the best way to learn how to marry novelty and monotony, harmonize imagination and rigour and extract essence from substance to produce valuable research. Many students require a Master's program to get a clear sense of this. Getting it from a Y3/Y4 undergrad is simply impossible.

From the "amazing work" student, you have either got:

1. A natural researcher, the rare combination of intuition, intelligence and good luck; or

2. Someone sacrificing their own ideas to devote their mind to validating yours. They hope someday to attain a position where they can finally do their own thing: but this is just another El Dorado.

If I were you, I'd advise your problem student to stop this research assistance work under you to focus on his undergrad work and on trying to find his own research perspective, if not finding himself. I sense that this chap has a way of charming people into believing in him, then using the same charm to diminish the usual consequence of letting someone down: this cannot be encouraged - it's bad for his life and for the patience of those around him.

More importantly, I would advise you to drop this mentoring work altogether.

A programme that selects potential researchers from a field of students who have only proven that they are good absorbers and synthesizers of knowledge found by others is not going to be successful. Experience shows that while good synthesizers may well become good teachers if they have a desire to enlighten young people, they never come up with innovative ideas by themselves - they just hijack those of others and vary the look of them. Nobody wins if you start injecting this kind of individual into a research environment. They will never leave an unsuitable work environment of their own accord - and they may well use their intelligence and charm to cause division there.

Answer 7

I want to add one thought to all the previous answers.

Although not your problem to.solve, be aware that underperformance can arise as a result of mental health ("MH") issues - problems that haven't come up before, or in the same way.

That's not uncommon at undergraduate level. A student has suffered from undiagnosed MH issues, ADHD, depression, trauma, BPD, autism spectrum/aspergers, take your choice. They now have to.self manage much more, in an environment with high pressure challenges and either away from home, without peer support, in a new environment, in a home-but-covid environment that leaves them dejected, something happens or happened maybe... and they just can't cope,or gradually things falter apart. The energy and skillset to handle it is low. Sometimes they know what's up but not where to get help, sometimes they don't really realise it themselves.

You might want to be aware of it and consider if there is a way you can check if they feel okay, if the issue is about the course, their study circumstances, or their personal/emotional circumstances.

Answer 8

This is really a management and leadership question. The student cannot perform unless he/she receives periodic feedback which serves to inform where effort should be increased or redirected. These intervals should be as short as possible so the student can immediately see how his redirected efforts are doing. For example if the student has received mentoring that he should increase his research efforts, in what direction those efforts should be directed, and what parameters should be set. At the next feedback session his work should be compared with what he mentor recommended at the previous session. The mentor should advise the student if minor corrections are needed and commend him for the work done in that period, wish him well for the next period. This periodic feedback serves to keep the student on track, allows him to know where his efforts should be focused, and gives ample time for correcting faults. If this is not done I feel it is the mentor's failure not the students.

Answer 9

Inform the student that because of the fact that them did not produce anything of value, your letter will reflect that and it is not going to help them. Tell them that them would be better off not listing their time with you on thier applications. If they persists in asking for a letter, send a short missive that summarizes the project and its outcomes but do not "evaluate". The institutions are not interested in you or your opinions, they are interested in what this person can do.