9

What is the inherent idea of having a recommendation letter/ "letter of reference" requirement in academia, especially while applying to grad schools?

I understand that if the person giving the recommendation letter is a bigshot or even fairly well-known, people would be ready to take his word regarding the applicant's caliber.

However, if the person recommending is previously unknown to the university you are applying to (of course, they may use Google to dig up some information, or they may not want to), what weightage does his word carry? If I am on the other side, I won't be inclined to trust the judgement of a person whom I don't know. What I would want to do is to judge the caliber of the applicant for myself, but that's a separate issue altogether. (Judge yourself, or trust some metrics...)

Now, in general, if "the other side" is a decent enough university, it will have applications coming from all parts of the world. Then, situation no. 2 (above) is more likely than situation no. 1. So, what is the intent of having a recommendation letter requirement in the second case?

  • 3
    Surely an opinion from someone you don't know is better than no information at all. Also, a really good letter will not just state the writer's opinion ("I think Jane Smith is awesome") but provide evidence, even if subjective evidence ("Jane Smith wrote an awesome paper in my class that showed a deep understanding of X, Y and Z"). – Nate Eldredge Oct 22 '14 at 5:57
  • 3
    @NateEldredge I think the question is still fair (and something that puzzles me as well) - why would an university care about what a prof. entirely unknown to them on the other side of the planet says about a student, no matter how fuzzy or concrete the information is? I "get" recommendations when everybody knows each other, or when the school actively approaches a prof. to rate an applicant, but if both of that is untrue I also kinda fail to see the value. – xLeitix Oct 22 '14 at 6:23
  • 2
    @NateEldredge Also, surely, an opinion from someone who is clearly not impartial and has presumably much more the interest of the applicant than the interest of the school in mind is not necessarily better than no information. – xLeitix Oct 22 '14 at 6:24
  • @xLeitix - (reg. comment no. 2 in particular) I'm glad you got my point precisely! When you don't know the guy who's writing the letter, you can't be sure about his "impartial"ity/motivation/interest. Then why bother about this formality at all? That was exactly what I intended to ask. Thanks :) – 299792458 Oct 22 '14 at 19:18
  • @NateEldredge - I like your sentence no. 1 and it commutes with the answer below. However (minor nitpick), I don't think it is fair to call it "evidence", in the spirit of xLeitix's comments, which are also (exactly) the spirit behind my question. It may be intended as an evidence from the recommender's side, but it can't really be deemed an evidence if you don't trust the guy's judgement. I think you take this into account when you write "subjective". – 299792458 Oct 22 '14 at 19:25
5

I agree with BrenBarn, but perhaps it will be useful to express things slightly differently.

To understand the purpose of letters of recommendation, it's important to think about the context. In my experience, the default assessment for a graduate school application is "not enough information" or "insufficiently compelling case." Most of the time, a rejection doesn't mean the committee felt there was enough evidence to prove the applicant was unworthy (although it can mean that for particularly bad applications). Instead, there just wasn't enough to justify admitting this applicant rather than the competition.

For example, grades are not very useful. The ceiling is low, the standards are inconsistent, and in any case getting good grades is a quite different skill from doing good research. Undergraduate research can be a more useful indicator, but it's still pretty limited. Some students have much better access to high-quality research opportunities than others do, so it's hardly a fair comparison. Plus many undergraduate research papers consist of straightforward work on specialized problems, done with considerable guidance and under some time pressure. That's a little closer to professional research than classwork is, but still not so close.

So the basic setting is that admissions committees are desperate for information. Judging research potential is really difficult, and it's at best loosely correlated with most of the hard data in graduate school applications. This is the context for letters of recommendation. If you have interacted closely with the applicant on a substantial undertaking in this field over a period of months or years, then you are in an excellent position to judge their suitability for graduate school. If you can convey this information to the admissions committee in a trustworthy and reliable way, then it can be far more valuable than anything else in the application.

Of course not all letters are useful. A letter saying "Joe got an A in my course" reveals nothing beyond what the committee could have learned from the transcript. More depressingly, some letter writers say substantive things but are not in a position to do so compellingly. If you are completely unknown to the committee, with no reputation or track record of prior students, then your letter will carry less weight (and even less if you don't at least have the excuse of being young).

This isn't as much of a problem as you might guess. Many people in the field have a reputation, even if they have never met anyone on the admissions committee, and they have an incentive not to hurt that reputation by writing foolish or biased letters. If necessary, someone on the committee can get in touch with them to ask further questions. Plus there are all sorts of opportunities for consistency checks (for example, if someone repeatedly says each year's top student is the best in years, that will be noticed).

However, there's still a genuine problem. A small fraction of applicants just aren't in a position to get compelling letters of recommendation, no matter how talented they are. They are going to be rejected through no fault of their own.

That's a sad outcome, but it seems to be unavoidable. If we had a more reliable way to judge research potential, we would eagerly use it. The point of letters isn't that they always help with the decision, but rather that they often help. When they don't help, the application joins the pile of rejections due to lack of information.

For comparison, one of the comments reads:

Now, ideal case, if A=B in ability, two possibilities arise - (1) If recommendation letter matters, then A gets picked over B (which is not fair),

Indeed, it's not fair, but it's impossible to gather enough information to make fair and reliable judgments in every case. Ultimately, the admissions committee has to accept that some wonderful applicants will be rejected because they couldn't prove how wonderful they were. (Not using letters of recommendation would reduce this type of unfairness, but at the cost of greatly reducing the information available to make good decisions in the other cases. It would amount to partially randomizing the decisions, which would not be in the department's best interests.)

To put it rather starkly, fairness is not the admissions committee's primary goal. Instead, the primary goal is to admit as strong an incoming class as possible. Letters of recommendation greatly help on average in achieving that goal, at the cost of disadvantaging certain applicants. This is a price departments are willing to pay.

  • Now, that's just what I was looking for! Thanks :) – 299792458 Oct 23 '14 at 4:55
  • For the sake of transparency, let me point out why I find this answer accept-worthy. Every caliber metric has its limitations, those of entrance tests, GPA's and GRE are fairly well known. Clearly, things are no different even for reference letters, something that I have always been skeptical about. Clearly, this metric is much more unreliable than others IMO (a very specific personal opinion). This answer doesn't pretend that we are living in a all-is-perfect-paradise, and just confirms my skepticism. – 299792458 Oct 23 '14 at 7:00
  • (contd.) I mean, specifically with this fragment "They are going to be rejected through no fault of their own. That's a sad outcome, but it seems to be unavoidable." My point exactly! This is why I find it accept-worthy. – 299792458 Oct 23 '14 at 7:01
10

At a minimum, a letter of recommendation is no worse than, say, a Yelp review. The person who rated Joe's Hamburger Shack 5 stars is a total stranger to you, but knowing that that person liked it is still more data than you previously had about how good the place is.

For a grad school application, there are few sources of concrete information. Hard data like grades, GPA, and GRE scores cover only a fraction of the relevant qualities of each applicant. It can certainly help if the admissions committee knows the recommender, but even if they don't, the mere fact that that the applicant is considered qualified by an active researcher at an accredited instituion is a significant gain over the other information available in the application.

Also, as Nate Eldredge mentioned in a comment, a good letter will not just state the belief that the applicant is qualified, but will give reasons. These reasons, again, come from a person who is presumed qualified to evaluate them, and are therefore more valuable.

It is true that if not only the recommender but also their institution is totally unknown to the committee (e.g., a recommendation from Prof. John Doe at East Podunk Community College), the information may not be helpful, and could even be viewed skeptically. But I don't think this is a common case. Even if no one on the faculty personally knows or has collaborated with any professors at, say, Harvard, you can bet that it means something to have a Harvard professor recommend someone. Although that's an extreme case, the same principle applies to other schools in varying degrees: even knowing that the school exists goes some way towards establishing credibility.

How much weight these are or should be given is debatable, but there is simply no other way to get the information in recommendation letters, namely a qualified professional's judgment of the applicant's quality. It may indeed be that in some cases the committee views the letters as unhelpful because they are too vague, but if many letters are informative, then vague letters are already "bad" in comparison, and provide a means of winnowing the applicant pool. Even in a hypothetical case in which only a few letters were helpful, the school doesn't really stand to lose anything by having them, and stands to gain useful insight that they can't get any other way.

Also, I would be interested to see some numbers on how often the admissions committee really does not know and has never heard of any of the people writing the letters. An important thing to consider is that in many cases one or more of the letter writers will have been consulted by the applicant to get advice on where to apply. Thus, there can be a self-selection process at work: professors know professors at other institutions, thus they advise their students to apply there, and then they write a letter of recommendation. I would guess that this significantly increases the likelihood that some of the letter writers will be known to the admissions committee. In effect, the same people are not only recommending the applicant to the school, but recommending the school to the applicant (by suggesting that they apply there).

A little edit: Based on your comments, it seems you are also worried about situations where, e.g., the letter writer is "biased" towards recommending the applicant. There are two responses to this: one is that everyone knows that. The recommender wouldn't be writing the letter if he were biased against the applicant. The recommendation is not supposed to be "impartial" in the sense that it mechnically assesses some attributes of the applicant. It's just supposed to be the honest opinion of a real person who interacted with the applicant in a real academic context.

More generally, there is simply no getting around the fact that virtually everything in academia operates on trust. When you submit an article to a journal, the editor has no way of knowing whether you plagiarized the text, falsified the data, etc. When you apply for a job and list dozens of publications on your CV, the search committee probably does not look up every single one ot make sure it really exists. For better or for worse, academic procedures are built on the notion of receiving "credentials" that indicate you are to be trusted. When someone receives a letter signed "Prof. John Doe, XYZ University", they assume that Prof. John Doe really is a professor at XYZ University. It is of course possible to fabricate such credentials, but to handle that problem would require a complete overhaul of academia. The basic reason people trust a letter-writer is because academics trust each other not to tell brazen lies about who they are, where they work, etc. They understand that the recommender may not be totally objective in his or her assessment of the candidate, but they nonetheless assume that the assessment is made in good faith by a qualified judge; that's all that matters.

  • 2
    As a letter writer, I'd say that about a quarter of the time I know someone in the department that the student is applying to (and they know me.) However, the admissions committee might be a much smaller group of faculty that doesn't include the people that I know in that department. I have sometimes gotten comments like "Send us more students like Joe- he's done really well in our program" from faculty that I know at other institutions. – Brian Borchers Oct 22 '14 at 14:51
  • I'll start by complimenting you on a very well crafted answer. Now, being more specific, I really like your full para 1. That itself (and alone) is "+1 and accept"worthy IMO. I feel that a better part of this answer pertains to the question's situation no. 1 ("known people"), and not to situation no. 2 ("strangers") which is what I was specifically referring to. In fact, in para 4, you only reiterate this point when you say "It is true ...()... be viewed skeptically." Then, the rest of the answer is intended to suggest that situation no. 2 is not "a common case" (contd.) – 299792458 Oct 22 '14 at 19:36
  • (contd.) That may or may not be true, and in any case, I don't have the numbers to claim either way (nor am I inclined to dig up these numbers, I'm not doing a survey!). But, even if it is less frequent, the possibility of such a situation arising can't be ruled out. Take for example (your example!) Harvard. It will get applications from all parts of the world, and it is clearly humanly impossible for the profs there to be acquainted with all profs in all these places, all over the globe. Thus, the possibility of situation no. 2 arising for real in academia can't be ruled out. – 299792458 Oct 22 '14 at 19:45
  • 1
    @New_new_newbie: Of course, but as I say in my answer, what is the downside? The worst thing that can happen is that you receive letters from people you never heard of. Well, so what? Even if only some of the letters are useful, that's better than nothing. It doesn't make sense to stop asking for letters just because some of them might not be helpful. – BrenBarn Oct 22 '14 at 19:48
  • True, but in that case, if applicant A was in situation 1, and B was in situation 2, the recommendation letter adds weight to A's application and B loses out on that. Now, ideal case, if A=B in ability, two possibilities arise - (1) If recommendation letter matters, then A gets picked over B (which is not fair), (2) If it doesn't matter, you have no way of picking one over the other. But if it doesn't matter, why ask for a recommendation in the first place? – 299792458 Oct 22 '14 at 20:00
1

I wholly agree with @BrenBarn's answer.

From the comments it seems clear that the question is also suggesting that there is a bias for a faculty member to write a good letter in the interest of the student over the interest of the other department. I would like to argue that this is faulty reasoning (even if it happens). I think this amounts to a little bit of game theory.

At the outset, we assume a faculty member has no relationship with the department s/he is writing the letter to. Assuming the faculty member writes a compelling letter, it may get noticed and there is some chance that that letter will lead to the acceptance of the student.

If the student is accepted and performs well, the faculty member's word will be more valued by the department during the next round of selecting PhD students and the faculty member may receive an email saying "Send us more students like Joe- he's done really well in our program" as per @Brian Borchers' comments. If the student is a flake, the faculty member's word is now (what is the word?).

This means that the next time that faculty writes a letter to that department it will be dismissed, even if it is compelling. Moreover, this may reduce the stature of the faculty member's entire department in the eyes of the other department.

So, in the end, I guess my answer is that to some extent faculty letters can establish a relationship between different departments. To the extent that they do, such letters are valuable.

  • +1 for the sincere effort and trying to enhance Bren's answer above. But again, why does everyone keep missing the point that I'm trying to make, which is applicant B (in situation no. 2) potentially losing out to applicant A (situation no. 1), just because A's institution/recommender was known to the department, while B's reference letter was viewed "skeptically" and hence, counted for nothing! – 299792458 Oct 23 '14 at 4:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.