15

Top math grad programs in the U.S. will receive tons of applications from people with excellent grades in grad or advanced undergrad math classes, high math subject GRE scores, and some research. No doubt there are other factors like competition results or awards from the student's department, but they tend to carry little weight, being considered as not very predictive of performance in math research. Not to say that doing well in a competition is useless, but the marginal predictive utility of doing well vs doing extremely well is insignificant in the sense that you're not gaining much info, if any at all, about future performance when comparing top 50 in Putnam and IMC to top 10. It will just be seen as the difference between solving constructed problems well and solving constructed problems very well by putting in even more effort practicing them.

Judging the research has its own problems with verifying how much the student learned and did on their own, so admissions committees turn to recommendation letters to distinguish between very good students who have nearly maxed out all the other criteria. However, recommendation letters are qualitative in nature and can only be justified on the basis that they provide info on a candidate's ability to succeed that other factors don't. In other words, recommendation letters aren't perfect but it would be much harder to make decisions without them just using other factors, which I agree with.

All that being said, why don't U.S. programs use entrance exams as a factor on par with classes, research, and letters? They shouldn't have the bulk of the weight as that may lead to similar situations as in India and China, where students go all out just for the exam at the expense of other skills, but there should at least be some weight. Something like the preliminary exams that colleges have for 1st and 2nd year students in linear algebra, abstract algebra, real analysis, differential equations etc. could be given to applicants instead of waiting for the students to enter and then requiring them to pass those exams, which is done at almost every university.

Another idea is interviews or oral exams, but the biggest reason why those may not be used is the amount of time required to administer interviews to 25-50 applicants who have passed all the checks on previous factors (good enough classes, grades, and letters) and compare those interviews. In comparison, the effort needed to write, proctor, and grade exams can be spread out. Staff outside the committee can proctor and grade while the committee just has to come up with the questions and look at the results.

9
  • 17
    In a faculty of around 50, there might be about 20 or so different fields represented. What would you put on such an exam? How would you validate it? Note that both false positives and false negatives are destructive to the enterprise.
    – Buffy
    Nov 7, 2022 at 21:41
  • 11
    Pretty similar to Can we stop looking at recommendation letters and rely purely on "objective" measurements during the admission process? - the ability to do well on some test is not well correlated to ability to think creatively and perform research. I'll also defer to Deming (Mr. Quality): "Just because you can't measure it doesn't mean you can't manage it." Objective measures really stink for evaluating people.
    – Jon Custer
    Nov 7, 2022 at 21:43
  • 17
    Having hired a reasonably large number of technical people over the last 20 years, I can state with great confidence that a 'objective' test would never have provided useful information about their future performance. Note that many universities are de-emphasizing (or eliminating) standardized tests (ACT, SAT, GRE). The complaint that the GRE isn't the 'right' test does nothing to argue that there is a 'right' test that is actually useful.
    – Jon Custer
    Nov 7, 2022 at 21:55
  • 6
    This doesn't strike me as a great question for this format, i.e. that has an objective or useful subjective answer.
    – user137975
    Nov 7, 2022 at 22:06
  • 2
    For what it's worth: top departments do positively view exemplary performance on the Putnam. I know someone accepted to Harvard grad school in large part because he placed top-100 in the Putnam. Nov 8, 2022 at 10:15

4 Answers 4

33

First of all, even just 3 years ago, administering such exams would have been a logistical nightmare. It takes me (and my students) 10 hours each way and $1200 in airfare to get to Cambridge, MA, and even longer (and more money) to get to, say, Charlottesville, VA. The alternative is to find and pay someone here to proctor one exam for one student, or maybe have two students drive 2 hours each way so that one person could proctor one exam for three students. (Now it's not a nightmare only if you trust exams proctored over Zoom.)

Second of all, there are two such exams. One is the GRE, though that suffers from being a multiple choice test and from having a syllabus that's somewhat outdated and narrow and also aimed at more of a mid-tier graduate program. A second is the Putnam Competition, though that is taken voluntarily and is perhaps too hard (the median score is sometimes 0, after all) even for top graduate programs; it perhaps also has too much emphasis on problem solving within a short time span. (Putnam scores definitely are considered by admissions committees.)

Third, top graduate programs are looking more for mathematical talent than mathematical knowledge. It's the experience of basically all graduate programs that a certain kind of ability to learn and do mathematics is much more important than knowledge of mathematics, because a sufficiently apt student can catch up on the necessary knowledge very quickly. June Huh, who just won a Fields Medal, did not study mathematics as an undergrad, almost failed out of undergrad in any case, and knew relatively little mathematics with relatively little classroom mathematics education when he was first admitted to graduate school. (He basically learned mathematics in a haphazard way by working as Hironaka's personal assistant.) There is no way he would have done well on any exam (perhaps even when he moved from his first graduate program to another graduate program, having proved a 40 year old conjecture), but of course every graduate program that rejected his application would now say that was a mistake, considering he has won a Fields Medal.

(Personally, I had well below average mathematical knowledge when I started graduate school, having taken a minimum number of courses for a math major at a liberal arts college, though I did write a nice undergraduate thesis.)

9
  • I did mention math competition scores aren't worthless, but see my remark about how the difference between a good and very good score doesn't indicate anything predictive. Good point about talent vs knowledge, but there can't possibly be a strong emphasis on talent in the way you describe it (an ability to learn and do new math). Otherwise people would just game the system by pretending not to know things and then "rapidly" learning under a professor via a class or project so that their talent and perceived ability to learn is inflated. Nov 7, 2022 at 23:02
  • I agree that the GRE is aimed at mid-tier programs, the knowledge tested is just the bare minimum for top 30 universities. Nov 7, 2022 at 23:07
  • 12
    @Displayname: I spend 30 minutes talking math with you and I will usually have a pretty good idea of your ability to learn mathematics. That's of course the better measure, not the change in your scores in something over a semester. Most applicants have some indication of this measure in at least one of their letters of recommendation. Nov 7, 2022 at 23:14
  • 1
    @Displayname: If I really want to have something that's completely ungameable, I make up some new set of axioms for a mathematical structure that has never been studied before, then show them to you and see how quickly and easily you grok those axioms and a few of their elementary consequences. (Of course to make my life easier I could just use some structure you likely haven't seen before; you would have to be a good actor to pretend not to have seen them before when you have.) Nov 7, 2022 at 23:34
  • 11
    @Displayname: If a student does research with me, this automatically happens at our first substantive meeting, because the research they're doing with me is on some mathematical structure they have never seen before. (This also tends to work for "research" - meaning independent discovery exercises on something known - since I have a collection of obscure topics for this purpose.) Nov 8, 2022 at 0:59
16

In brief, in my 40 years of involvement with graduate admissions at my R1, ...

... because the U.S. educational system does not substantially prepare people for grad school in mathematics (REUs are really just fluff, most times). So there's no "content-related" exam that could be given that would reflect peoples' future potential...

Sure, this aspect of the U.S. educational system makes most students look worse by comparison to nearly all the other educational systems in the world, which do not emphasize "broadness" at all... quite the contrary.

Nevertheless, in my long-term observation, this initial advantage-or-not usually disappears within one or two years!!! (... of grad school).

So, in terms of "pseudo-factual things would could pseudo-objectively test"... they'd mostly be irrelevant.

Seriously, the "intangibles" in letters of recommendation are the most relevant, yet least "objective", things that predict success in grad school.

1
  • 4
    I went to a small liberal arts college for undergrad and didn't take any graduate courses. I always felt behind...I'm not sure the disadvantage ever goes away.
    – cgb5436
    Nov 8, 2022 at 2:45
9

Some departments actually do use an entrance exam: The Math subject GRE exam.

But in the end, the question everything comes down to is this: How predictive of success would such an exam be? Would students who do well on these exams do better than the pool currently considered for admission? People and universities have, over the decades, come up with all sorts of ideas of how better to assess how successful students will be in graduate school (or when companies are hiring, or for faculty hiring, or for undergraduate admission, or ...) and the general observation is that it is just very difficult to predict, especially the future. If there was a single criterion we could apply to predict how successful a student would be in graduate school, we'd all be using that criterion.

8
  • GRE doesn't count, I mentioned it in my 1st paragraph as one of the factors that all very good students have maxed out. I had in mind an exam that universities can tailor to their own desires, testing the knowledge and skills (to the extent they can be tested through problems) that they would actually care for students to have. Nov 7, 2022 at 21:36
  • 8
    Math part of GRE != math subject GRE
    – TimRias
    Nov 7, 2022 at 22:40
  • 10
    @Displayname Do you have a source for the assertion that "all very good students have maxed out the math subject GRE", along with a non-circular definition for what it means to be a "very good student"?
    – Bryan Krause
    Nov 7, 2022 at 22:48
  • 5
    @Displayname I don't know what your comparison group is, but at a typical state school, the fraction of applicants who have a perfect Math Subject GRE score is for all practical purposes zero. From wikipedia: "The mean score for all test takers from July 1, 2011 to June 30, 2014 was 659, with a standard deviation of 137." A perfect score would be 990, but the 99th percentile is at 920, so it is just not true that "very good students max out the GRE subject score". Nov 8, 2022 at 16:30
  • 5
    It used to be the case years ago that a large fraction of the scores (something like 10% as I recall) on the GRE subject test in mathematics were the top score possible. However, in recent years this has changed. An ETS report shows that the 97th percentile is at 960 and the highest possible score is 990 (although the percentiles are only reported up to 960.) ets.org/content/dam/ets-org/pdfs/gre/gre-guide-table-2.pdf Nov 8, 2022 at 16:36
5

My department (Chemistry) used to use standardized American Chemical Society entrance exams in the (then) four main chemistry sub-disciplines: analytical, organic, inorganic and physical. Students performing below pre-determined thresholds had to take one or more remedial courses. Then biological chemistry came along, and faculty decided that taking five entrance exams was simply too much.

This problem was resolved when faculty in the various sub-disciplines decided to drop the entrance exams. One by one, the entrance exams were dropped until only my own area was giving entrance exams. The last straw was when faculty in my sub-discipline found out that other faculty were using results on our entrance exam to make decisions about the new graduate students! So we dropped the exams as well.

Overall, I think the entrance exams had some utility is assessing fairly minor deficiencies, but not enough utility to justify keeping them. Dropping the exams caused no significant issues.

4
  • Interesting, but why did the various subdisciplines decide to drop exams? I don't even quite understand why other faculty using your exams was a reason for you to drop yours---did you have to administer them yourselves?
    – Kimball
    Nov 8, 2022 at 13:31
  • We had to administer the exams and grade them, during the hectic week before classes started. New graduate students arrived a week before the semester’s start and went through a fairly intensive orientation. It got to be too much for the marginal benefit. As for the other subdisciplines using our exam results, it just made no sense: a student might be perfectly suited to earn a PhD in organic chemistry, but terrible in physical chemistry. Or vice versa, just to pick two areas. So we felt it subverted the rationale for giving the exams in the first place.
    – Ed V
    Nov 8, 2022 at 14:00
  • I'm confused---do you mean placement exams for students who have already been admitted, rather than entrance exams to decide admissions for grad school applicants? That is what this question is about.
    – Kimball
    Nov 8, 2022 at 14:14
  • 2
    Placement exams actually. There was no possible way, back then, to administer exams to the 300-350 students who applied each year. And, yes, the question was about entrance exams per se, but it seems to be a distinction without a difference. Exams just do not accomplish what it is hoped they might accomplish.
    – Ed V
    Nov 8, 2022 at 14:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .