91

I did my undergraduate degree in the US and am heading to graduate school here in less than a month, so I myself have taken the Graduate Record Examinations (the general as well as two subjects tests) and I guess it always just seemed as a sort of un-avoidable formality that nothing could be done about, and so I just took it and got it over with.

In the intervening year between undergrad and the start of my Ph.D. work, however, I traveled overseas to Cambridge where I found that I was quite mistaken: the GRE is very, very avoidable. The solution is simple: don't apply to universities in the US.

By the time I had arrived there, I had already gotten it over with myself, but for most of my peers there, this was not the case, and quite a few of them had simply decided to not even bother applying to the US because of the inconvenience that comes along with that in the form of the GRE.

This made me wonder: are admissions committees at US universities aware of the number of highly qualified candidates they miss out on because of the GRE?

I could understand being willing to miss out on the potential recruitment of these students if the GRE were a significant part of one's application, but I have yet to find any US professor tell me that the GRE scores are weighted highly when it comes to making admissions decisions (perhaps I just haven't asked around enough?). In fact, I've often been told it's the least important factor when deciding whether someone should be admitted. (Indeed, my impression is that the general GRE is more or less a joke and only serves as a convenient way of tossing out applicants who would have been found un-qualified for other reasons.)

Putting aside for a moment the issue of those who decide not to apply to US universities, let's consider the inconvenience faced by those who do. Once again, if you're from the US, I can imagine simply not being aware of this (I know I wasn't), but I now know of several people who have had to fly (sometimes the flights have even been inter-continental!) in order to sit to take a GRE test. And even for those who don't (like probably most of us in the US), there is the ridiculous price: almost $200 for the general and an extra $150 per subject test. I was under the impression that admissions committees encourage people from all backgrounds to apply, rich or poor, but how can they honestly expect this to happen if even those who don't have to fly have to shell out anywhere from $300-$500 in addition to the application fee? (I personally find it a bit nuts that these tests cost several times more than the application itself.)

So, could somebody please explain to me why we still require students to take these things? Do they really add information about the applicant and their abilities that could not be found out any other way?

  • 12
    Now that I think about it, perhaps a more appropriate sub-title would have been: "Why did this ever exist?" =P – Jonathan Gleason Aug 12 '13 at 1:16
  • 13
    There is also the situation where one has to wonder whether to shell out an extra $180 to squeeze a couple extra percentile from the first time. I loathe giving the ETS more money for a test that predicts success in graduate school by only testing the ability to do elementary calculus quickly. – DCT Aug 12 '13 at 1:25
  • 23
    The solution is simple: don't apply to universities in the US. — Or apply to departments in the US, like mine, that don't require GRE scores. – JeffE Aug 12 '13 at 3:48
  • 10
    The GRE "only serves as a convenient way of tossing out applicants who would have been found un-qualified for other reasons": That may be. But well-known departments get lots of applications, and the admissions committee is made up of faculty who are very busy already. So it's understandable that they'd grasp at a quick way to weed out the least qualified applicants. I'm not necessarily defending the GRE, but perhaps you can see why departments might use it. – Nate Eldredge Aug 12 '13 at 15:32
  • 14
    GRE still exists because someone is making profit out of it. – kaptan Jul 30 '14 at 18:53
56

Empirical evidence on the relationship between GRE scores and post-graduate performance

There is a massive meta-analysis by Kuncel et al (2001) that empirically evaluates the correlations between various aspects of the GRE with multiple post-graduate performance criteria.

Based on hundreds of studies and thousands of participants, GRE shows reasonable correlations with post-graduate GPA (i.e., around observed r = .21 to r=.43). Similar correlations emerged between GRE and faculty ratings of the student.

Correlations of GRE with research productivity and publication citation counts were smaller, but still positive. This is not surprising given that these are more distal outcomes and there are many non-academic reasons why people may or may not pursue an academic career or have publishable results.

The authors thus concluded that GRE was a valid predictor of a wide range of graduate outcomes. They also noted that "subject tests tended to be better predictors than the verbal, quantitative, and analytical tests."

In general, selection decisions are assisted by standardisation, and a big part of academic achievement involves measuring baseline ability. Thus, the GRE combines both standardisation and competence measurement.

Response to comments

There have been a few points made in other answers and in comments, which I'll comment on here:

  • Conflict of interest: @msw wrote that they "consider the cited paper to be junk because ETS provided all available data". In general, I don't find the results in the meta-analysis surprising. Most tests like the GRE tend to have fairly strong correlations with general cognitive ability. It is well established through thousands of independent studies that IQ scores correlate fairly well with both school grades and job performance (i.e., in the r=.50 range; see Neisser et al 1996 for a field consensus review). The results show correlations less than .50, but that's not surprising given some of the issues around standardisation, practice, range restriction, domain specificity and so on.
  • Belief there is no correlation based on personal observation: Note that if the correlation is around .20, that means that 4% of variance has been explained. That leaves a huge amount of variance in performance to still be explained. It would not be surprising to meet many people that did well on GRE and poor in graduate school or vice versa. Thus, it is problematic to rely on personal experience when it comes to evaluating the validity of tests where such correlations are likely to be only modest.
  • Small correlations are useful: While a .20 correlation is small, it can still help make selection decisions. In particular, when evaluating the suitability of selection tools, you need to contrast the validity of a given tool with other available tools (e.g., interviews, GPA, references, and so on). I'm not as familiar with results in the graduate selection domain, but certainly in the employee selection domain, which is quite analogous, cognitive ability tests tend to correlate more highly than interviews, references and so on (for a comprehensive meta analysis of employee selection, see Schmidt and Hunter, 1998). That said, the best selection decisions are typically obtained by integrating multiple selection tools. Furthermore, the small correlation also should highlight to individuals who score poorly on the GRE that it is not that predictive, and therefore it shouldn't discourage an individual from pursing post-graduate study.
  • Does training invalidate the GRE?: @user8134 wrote "many people significantly improve their GRE scores by taking courses with prepping companies like Kaplan. So it seems GRE test does not measure any intrinsic ability/talent necessary for grad. school." I agree that individual differences in training and preparation for the GRE may influence test scores. That said, if you characterise test scores to be determined by true ability, training, and error variance, then I would expect that true ability would remain the much larger source of explanation in test scores. This is based on general observations about testing for ability based assessment. In general, the degree to which training is an issue would depend on how much the test materials teach to the specific test. Overall, I would assume that this would reduce the potential validity of GRE, but that the GRE would still be useful. Furthermore, training and nuisance factors can be used to do better on many selection instruments. For example, people can be coached on how to frame their CV or how to answer questions in interviews. Such training is potentially a source of error variance, but it doesn't invalidate CVs and interviews completely.
  • Why would anyone care about post-graduate GPA? (@JeffE) @JeffE further notes " in PhD programs, the only thing that really matters is the student's research output.". Some post-graduate courses include meaningful graded coursework and others don't. For the courses that do include meaningful coursework, then such coursework provides a more standardised way of measuring post-graduate performance. So the validity of GRE in predicting such outcomes is not surprising. And thus, presumably we could generalise this to being indicative of how people perform in less standardised aspects of post-graduate performance. Of course, there's an inferential leap here, but in general, performance in related domains tend to correlate (e.g., coursework in mathematics with research performance in mathematics); it's not perfect, but it's still a positive correlation. Furthermore, if the validation study includes some post-graduate coursework where everyone gets top marks and such data is mixed with studies where grades are valid measures of performance, this would only serve to attenuate the observed correlation. Thus, this would suggest that the correlation between GRE and post-graduate GPA is higher than reported by the meta-analysis. Also, the meta-analysis does report correlations with research output and they are weaker but still positive. It also reports correlations with supervisor ratings.
  • Better alternatives to GRE: None of my comments above are necessarily advocating the use of the GRE. Developing an effective selection and recruitment system whether it be for employment or post-graduate admission is a complex task. That said, most post-graduate selection systems would want to get a reliable and valid measure of academic aptitude. GRE, IQ tests, other ability tests, undergraduate GPA, all have reasonably validity evidence. And in general standardisation and efficiency are important. So, for example, administering your own selection tools takes more time, whereas taking pre-existing measures like GPA and GRE is more efficient.
  • Ethics of requiring applicants pay money to complete GRE: Several people are critical of the GRE on the basis that it costs several hundred dollars to complete. I think that this is a perfectly legitimate question, but that the question of predictive validity can be answered separately. Such a fee could potentially discriminate against low income applicants. That said, presumably the fee in comparison to forgone wages associated with completing a post-graduate degree is fairly small.

References

  • Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2001). A comprehensive meta-analysis of the predictive validity of the graduate record examinations: implications for graduate student selection and performance. Psychological bulletin, 127(1), 162. PDF
  • Neisser, U., Boodoo, G., Bouchard Jr, T. J., Boykin, A. W., Brody, N., Ceci, S. J., ... & Urbina, S. (1996). Intelligence: Knowns and unknowns. American psychologist, 51(2), 77. PDF
  • Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological bulletin, 124(2), 262. PDF
  • 30
    Why would anyone care about "post-graduate GPA"? – JeffE Aug 12 '13 at 3:49
  • 10
    Indeed, after my first year of grad school, I remember talking with my father about this. He was shocked. "You mean you get an A just for attendance?!" I stopped to ponder how many of my friends reliably attended their classes. "Well, actually, ..." – Anonymous Aug 12 '13 at 3:53
  • 15
    I consider the cited paper to be junk because "ETS provided all available data on GRE validity to us in unaggregated form." and the notoriously closed ETS doesn't give data often. The second author's long-time commercial affiliations further darken this paper's intellectual independence. – msw Aug 12 '13 at 12:57
  • 8
    No doubt there is going to be some correlation between GRE scores and success in graduate school (just like there will be some correlation between success in graduate school and timed arithmetic tests). The question is if it's worth the price. It isn't possible to implement this, but one way to test would be if the students took the tests for free, and the graduate schools had to pay in order to see the scores (without just increasing the application fee, and suppose they could decide whether to pay after reading the rest of the application). – DCT Aug 13 '13 at 4:49
  • 7
    Note that this may just be an example of self-fulfilling prophecy. You can probably make (m)any a measurement correlate with a-posteriori quality by convincing everybody to select based on that measurement. Better schools pick those with better values and, on average, produce better output -- surprise. – Raphael Jul 30 '14 at 15:52
47

This is a good question. I have not been a fan of the GRE for 20+ years, although (through whatever luck I had a good-enough number on it myself that it didn't harm me...) many are. Having been on admissions committees and very much involved with graduate programs for 30+ years, I've had ample opportunity to see the (non-) correlation of success in graduate mathematics with GRE subject-test scores. (The other parts are often useful as tests of English fluency, mainly.)

Of course, the world would be a simpler place if GRE subject-test scores really could show talent for higher mathematics. We note that the Educational Testing Service (in NJ, that makes the GRE and other stuff) is a for-profit that has a vested interest in maintaining its products' apparent importance.

The cost is unfortunate, and certainly discriminates against people whose currency doesn't compete well with USD.

The vaguely useful bit of information provided by GRE is that, well, yes, it is the only thing that most applicants will have done, thus, if one insists on "simple comparisons", it is the only thing that allows that. It is clear that comparison of GPAs is even more pointless.

I have known admissions committees that simply ranked applicants by GRE subject test score. There! Done! :)

No, I do not care very much about GRE numbers, but in a way I am glad that some admissions committees do, in a fashion to the way that under-valued stocks are good investment values.

I do think that the elite graduate programs use GRE subject test as a convenient filter, because it selects somewhat for "quickness/cleverness", and they can afford to "lose" some prospects, because they have so many who are "quick/clever".

In the U.S., having a GRE subject test score is also a sign of awareness that people expect you to take it. Thus, it doesn't matter so much what one's score is, but that one _is_aware_ ... even if it is only of "expectations".

But, in summary, no, I see no point in it. But there are economic incentives for ETS to keep making money. And a great number of admissions committees in math have personal predilections that lead them to be fond of (over-) simple numerical quantification, so... there-we-are.

Edit: As @msw observes in a comment, indeed, if GRE measured significant academic achievement cumulative over several years... it would be odd that one could usefully do the "prep" courses ETS provides. :)

Yes, performance on GRE probably is a good indicator of how well a kid can do on a multi-hour, timed, multiple-choice test, etc. Yes, if we make subsequent coursework resemble this (!?) then we give the GRE predictive power. No, I do not recommend making everything multiple-choice! But, amazingly, some people do believe that this could be done, and purportedly save us all a lot of work.

Sure, these things measure something, and produce numbers that can be manipulated. There are people who are inexorably drawn to the possibility of making final decisions in those terms, even when the significance of the numbers is unclear. Meanwhile, reading letters of recommendation and personal statements is obviously not easily quantifiable. Of course!

If it were really the case that "standardized testing" could tell what its promoters like to insinuate, it would be convenient, indeed. But, again, some decades of experience indicate that these tests do not indicate whether or not people can sustain interest over 4+ years, work hard for 4+ years, continue to develop scientific sensibility, and so on. And the latter issues prove to be vastly more important for completion of a PhD. In direct observation of about 700+ grad students, I'd estimate that fewer than 20 dropped out or failed due to lack of intellectual capacity or lack of prior knowledge. Rather, loss of interest in the subject, or personal issues (mental/physical health) dominate. This "sample" of mine includes a very wide range of GRE scores and even GPA.

More anecdotally, several specific examples stick in my mind, of very low percentile on GRE subject test (bottom 10 percent or smaller...) but exceptional achievement in coursework, prelims, and thesis work. These peoples' potential was easily visible in letters of recommendation and personal statement.

  • 29
    More bluntly, it's a self-perpetuating racket by ETS who makes a bloody fortune on useless testing and selling "preparation materials" to train the subjects of their supposedly valid testing methods. Deductive logic test: if the GRE is a measure of inherent qualities, then preparation should not significantly affect a subject's score. Ⓐ True Ⓑ False Ⓒ Either way, ETS makes more money. – msw Aug 12 '13 at 12:29
  • 1
    If it matters, I did just fine on my GRE with no preparation, tyvm. – msw Aug 12 '13 at 12:30
  • 5
    @anon, Oh, yes, indeed, might be. Or not. SAT scores certainly indicate even less than GRE subject test. All these multiple-choice tests do substantially test test-taking ability, speed, and especially the math parts are very speed-puzzle-oriented. It's not a bad thing to be quick, to have the capacity to be undeterred by puzzles and tricks, but that capacity itself is not much related to serious mathematics, from my perspective. – paul garrett Feb 15 '14 at 23:16
  • 1
    @anon, believe me, I know what can happen. High grades in the usual curriculum, which itself is somewhat antithetical to genuine mathematics, do not move me particularly either. The standard undergrad courses based on standard texts are quite often ghastly. Thus, e.g., students who are attracted to that raise suspicion in my mind that they'll not like genuine mathematics so much. – paul garrett Feb 16 '14 at 4:46
  • 1
    When I was on my department's graduate admissions committee, I found GRE scores to be somewhat useful in the case of students from obscure (often foreign) schools where we didn't know the standards well enough to extract useful information from the grades and where the letters of recommendation all look (to westerners like me) isomorphic. – Andreas Blass Dec 12 '14 at 0:12
23

In my department, GRE scores are used mostly as calibration for students who have good grades but are from universities/programs that we are not familiar with and whose quality we are therefore not sure about. For students with good credentials from strong programs the GRE is pretty much irrelevant.

  • So, GRE shouldn't be compulsory for all schools but only for those schools which are unknown. – user774025 Aug 12 '13 at 10:24
  • 15
    @user774025: I'm sure a policy like that would violate all sorts of equal opportunity rules. Requiring GRE only for those from unknown schools is effectively putting up an economic barrier to students from those schools. – Willie Wong Aug 12 '13 at 12:33
  • 2
    I suspect that many schools which list the GRE as compulsory are willing to take applications without it ... write to the admissions office with this question. – Peter Shor Aug 17 '13 at 19:03
16

A couple of issues:

Re Mr. Anglim's comment, a suspect issue re. any correlation between GRE scores and success in grad. school is that many people significantly improve their GRE scores by taking courses with prepping companies like Kaplan. So it seems GRE test does not measure any intrinsic ability/talent necessary for grad. school.

Another issue is that the GRE is a form of forced labor: ETS uses one of the sections in the exam (just which section is unknown to the test-taker) as data for future exams, i.e., the section is not counted for the score of the test. So one is expected , basically, to work for ETS for free, producing high-quality data they would have to pay a lot for, or may not be able to produce themselves. GRE also puts out books to prepare for the exams, which cost above $20 each.

Now, ETS could find a way around this by asking, say, "There may be a section in this test which we use as data for future exams: if there is one, would you be willing to take it, or do you prefer to skip it?". This - asking you to work for them for free - is unethical, IMO, if not illegal. And ETS' BS response to this (I called them) is to tell you: "Well, if you disagree, don't take the test." The problem is some programs require you to take it in order to apply for their grad. programs; ETS is the Frank Burns of testing.

  • 2
    They also demand of you that you will say nothing about what was on the test afterwards, which IMHO is even worse. – Anonymous Aug 12 '13 at 12:58
  • Interesting answer. I was not aware about ETS using part of the exam for data. Could I trouble you to add a relevant link about this? Thanks. Also, I like the Frank Burns metaphor too, but which aspect of Frank are you specifically alluding to here? Or are using him generically as a well known (in American popular culture) scuzzball? – Faheem Mitha Aug 14 '13 at 16:09
  • One could use "Montgomery Burns" instead, and I think the metaphor holds just as well. – J.R. Aug 17 '13 at 19:34
12

@Jonathan, if you were to ever come to the other side, you will see a great number of glowing applications from top students of the top university in Bolivia or Madagascar. How do they fare compared to the top student of the top university of Idaho? Or a top student from a mediocre university in Massachusetts? If you admit grad students planning to use them as TAs, how do you know that this student from Nepal will be understood by your students in Tennessee, who heartily laugh at both New York and Californian accents?

The GRE fills the role of such a filter, and as such is the cheapest, easiest to use tool available to graduate schools in the US. If you can get any UNESCO money to design and implement a version of it that would be free to international applicants, maybe you can get a Nobels fredspris for your efforts. (This is not such an unrealistic idea as it sounds: an alternative, freely available operating system known as Linux has been developed by enthusiasts, and in many dimensions has replaced UNIX.)

  • 3
    My department's strategy for applicants from unknown schools (either foreign or domestic) is to admit one student with an apparently glowing record and then watch how well they do. After that, we have some basis for calibrating their grades and letters. Sadly, often the result is that we no longer consider applicants from that school, but sometimes it's the opposite. As for the accent, we use the TOEFL to judge English fluency, despite its similar problems. – JeffE Aug 12 '13 at 23:29
  • 4
    Also, I grew up just south of Nashville; I've never heard anyone laugh at either New York or California accents. – JeffE Aug 12 '13 at 23:32
  • @JeffE After that, we have some basis for calibrating their grades and letters. Your department is using person A's performance to figure out person B's potential. – scaaahu Aug 13 '13 at 4:14
  • @scaahu: Yes, that's right. We're using the performance of student A from school X to judge the potential of student B from school X (along with B's statement, letters, and grades, of course). – JeffE Aug 13 '13 at 20:44
  • 1
    @scaahu: in statistics terms, JeffE (and MANY, MANY other departments) try to gauge the conditional distribution of student B's ability given what they know about the grade distribution of the home university Z common to both students A and B, and basically are applying Bayes theorem to gauge the probability of student B's success in the program. (Bayes theorem underlies most of the data-driven decision making, from spam detection to terrorist identification in the airports.) – StasK Aug 13 '13 at 20:57
8

In a word, standardization.

I once asked this question to a colleague, and I appreciated the response I received.

Essentially, the faculty member told me, "What I like about the GRE is that it's the only way I can compare apples to apples."

He went on to explain, "How can I compare a 3.2 GPA at University X with a 3.7 GPA at University Y? I can't. But, at least with GRE scores, I can compare the two students on an even playing field."

My retort was that a high GRE score isn't necessarily a good indicator of potential in graduate school, although it might be a good indicator to determine the amount of time a prospective student spent preparing for the test. (I got a lot of milage out of my How to Ace the GRE practice book.)

He readily agreed, and assured me that it's just one piece of the puzzle. An admissions office only has so much to go on: a transcript, a GRE score, and perhaps a "Why I want to go to graduate school" cover letter.

We could throw out the GRE, but then there would be that much less information to base admissions decisions on.

My gut tells me that the negatives include the expense and hassle for the applicant, and a limited ability to predict how the student will actually perform. But I must admit my colleague had a point with his "apples to apples" perspective.

  • 4
    I can compare the two students on an even playing field — Except, of course, that you can't, especially for international students. At least in my experience, baseline performance on the GRE (and other standardized tests) varies significantly from country to country (and from social class to social class) because of the relative availability and expense of test-training. – JeffE Aug 17 '13 at 2:48
  • 2
    @JeffE: You and I agree on that point, although I will say that, insofar as "expense" goes, I only invested $20 or so at Barnes and Noble, and that did the trick. It's possible for most folks to give themselves a pretty good prep experience on a relatively low budget, provided they're willing to invest the time. – J.R. Aug 17 '13 at 19:32
0

Explains the quality of our PhDs, doesn't it. I have personally found the American education system far more wrapped up in silly and actually dangerous formalities than schools in Europe, generally speaking. I was surprised to find this out first hand because the common consensus is that Europe suffers from a great deal of pointless formality that the US does not. The problem is that it's a category mistake: elaborate bureaucratic formalities: yes; greater social formality: sometimes, depends; educational formality, not necessarily. Also, there is a difference between formality and rigidity. While titles are more important in Europe when addressing one another, they actually can enforce a very healthy relationship between the student and the professor which enables a healthy flexibility and informality to emerge. The casual "buddy" culture of the US is actually too chummy which encourages disrespect and I think the reaction to it is a certain kind of seething rigidity that manifests in the student-professor relationship which is unhealthy and domineering many times. But I digress. When I experienced American education first hand, it seemed infantile in its execution and the GREs belong to this set of things which contribute to the culture of pettiness (other things include the "publish or perish" doctrine which has resulted in the explosion of BS and CV padding nonsense and the "career academic" (hopefully that translates into English properly). While there are entrance exams in Europe, graduate school acceptance is according to other criteria. It's not always optimal, but certainly less irritating than this test taking nonsense.

Btw, I find the correlations mentioned above to be exactly the problem. They make the same mistake of drawing rigid and often erroneous inferences from a very poor selection of data.

I find that university education everywhere have been ruined by the pressure to put everyone through it. What you have are glorified trade schools in most cases with millions of applicants pushing through as if through a military entrance exam where narrow indicators sacrifice complexity for measures of simple routine. It's highly corporate and prone to the rat race. Graduate school should be the last place where this kind of pettiness manifests, and yet...

Summary: GREs are a result of the culture, of modern (American) university culture like the silly innovations of the German university which sought to label and shelve every person for particular slots in the machine of the the Reich.

  • 11
    -1: Godwin's Law. – Nate Eldredge Aug 12 '13 at 15:36
  • 6
    @NateEldredge Heh! Good point, but, nevertheless, this answer does make some good points about culture-centric deconstruction. – paul garrett Aug 12 '13 at 15:43
  • 1
    Flagged. There might be some good points here, but I can't see them through the rant. – JeffE Aug 12 '13 at 23:34
  • 7
    Upvoted. Personally, I enjoy a good rant, and there is much to rant about wrt university systems. Robert forgot to mention that PhD was a German invention (originating at Humboldt University of Berlin) and spread like a disease across the world. Researchers got along just fine before it. The point about US universities becoming a mass system is also well taken. – Faheem Mitha Aug 14 '13 at 15:57

protected by Wrzlprmft May 13 '17 at 6:36

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.