18

Multiple choice questions are nowadays very common in many of the world's institutes. Particularly in entrance tests, they use multiple choice testing. Why are these academic institutions now looking to judge their students using multiple choice tests?

  • 1
    They are not looking to judge the students. They are looking to assess their students' knowledge. – JeffE Mar 31 '16 at 15:58
  • Comments are for clarifying details or pointing out problems in the question. Comments are not for posting answers, or debating the usefulness of multiple choice exams - but feel free to discuss at length in Academia Chat. – ff524 Apr 1 '16 at 17:48
  • @JeffE good point sir – dev_shanghai Apr 1 '16 at 17:52
36

Because they are graded easily and fast, and there is no room for subjectivity.

The answer to question 1 is option b). If you marked b), you get points, if you didn't, you are wrong. The grader doesn't even need to know what the question is about, just that the right answer is b). Also, there is no middle ground, no partial credit, and no subjectivity involved.

Of course, there are a whole lot of problems with this system: poorly phrased questions, the student doesn't show the process, just the final result...

| improve this answer | |
  • 24
    I would have expected an answer of the form "(a) it is cheaper, (b) can be graded fast, ... (e) all of the above" – Hagen von Eitzen Mar 31 '16 at 14:49
  • 7
    To add - In a world where you can get sued for discrimination against anything, it's best to have the cold, calculating, heartless computers handle the scoring (as @Davidmh said, no room for subjectivity). It takes potential legal issues out of the equation, or at least gives them a lot more ground to stand on. Nobody can claim "I was descriminated against because there aren't enough 'x' in your review panel so I, as an 'x', am graded against a bias!" and other such nonsense. – Ethan The Brave Mar 31 '16 at 15:10
  • 1
    Just to be fair a lot of schools do have the concept of partial credits with MC tests where students can afterwards complain about badly phrased questions and the marking scheme is adjusted accordingly (questions being dropped entirely, multiple answers considered correct or some of the answers receiving partial credit (in cases where it's a correct answer, but not the most correct answer)). – David Mulder Mar 31 '16 at 20:45
  • 2
    @warren Unfortunately (at least in my experince in the UK) grad students learn the mindset that "using software created by somebody else" doesn't count as "academic work". Spending months re-inventing a square wheel that only turns once per minute counts as part of the credit for your thesis, but buying a wheel that works properly for 50 cents, or citing the use of somebody else's free open source software, doesn't count for anything. – alephzero Mar 31 '16 at 23:49
  • 1
    @warren I was referring to earmarked funding. The grad students are paid out of a different budget. – Jørgen Fogh Apr 1 '16 at 7:08
16

Multiple Choice Questions are nowadays very common in many of the world's institutes, particularly in entrance tests they conduct MCQs for the students. Why are these academic institution now looking to judge their students in MCQ tests?

In my experience (graduate admission to mathematics departments in the U.S.), multiple choice tests are not particularly valued or respected. People use them because of a lack of better options, not because these tests are considered a suitable measure of student accomplishments.

The problem is that course content and grading standards vary substantially between institutions. This makes it difficult to compare applicants to graduate school: how does an A- in a beginning graduate course at the University of the Middle of Nowhere compare with a B+ in an advanced undergraduate course at Caltech? Is it better to be the 4th strongest math major in your class at Princeton, or the best student in 25 years at Middle of Nowhere? For applicants from prestigious universities, letters of recommendation and comparisons with applicants in past years can provide plenty of information, but this doesn't work so well for applicants from less prestigious universities. Standardized tests help provide an objective measure that can level the playing field. (Of course they don't level it completely, but an applicant from the middle of nowhere who gets outstanding GRE scores attracts a lot more attention than one with mediocre GRE scores.)

Keep in mind there are a lot of colleges and universities out there, roughly 2500 in the U.S. alone. This means in a typical year, about a hundred students in the U.S. will be the best mathematics major to graduate from their department in the last 25 years. Some of these students are amazing, but in some cases it's not so impressive, since the department just doesn't get strong students. There has to be some measure beyond comparison with unknown classmates.

The reason why multiple choice tests are used (as opposed to essay questions) is because they are what's out there. If the GRE subject exam involved essay questions, that would be objectively better, but it would be nightmare to grade. For comparison, grading the Putnam exam takes a tremendous amount of work, and I'd estimate that an essay-based GRE would take at least ten times as many person-hours to grade, maybe more. I don't think anyone cares enough to want to develop the test, organize the test administration and grading, and pay for everything (with a combination of high test fees or outside funding). It's just not clear that it would be worthwhile.

| improve this answer | |
  • A standard test is not the only solution to the plethora of universities: in Europe we are managing quite well without it; mostly by interviews and reading project reports. – Davidmh Mar 31 '16 at 15:24
  • 1
    @Davidmh: But interviews would pretty obviously discriminate against people who are not good at face-to-face interactions. – jamesqf Mar 31 '16 at 16:41
  • 8
    @jamesqf - I'm playing devil's advocate here, but for graduate school, that might be a trait that a department would want to discriminate against. – thomij Mar 31 '16 at 16:47
  • 1
    This seems basically on the right track to me, +1. However, I think it overstates the barriers to creating tests that are standardized but not multiple-choice (MC) in format. Big testing companies such as DRC/CTB and College Board have the resources and technical expertise needed in order to standardize a non-MC test. They simply charge more money for it. The trouble is that many of their customers, e.g., the California public school system, are extremely cost-conscious. – user1482 Mar 31 '16 at 22:51
  • 1
    @G.Bach - no doubt that is true (ie "it's easier" is not necessarily a good argument). However, the vast majority educational institutions are not [currently] staffed to manage any other means of entry.The cost of those extra admissions personnel is likely a huge factor in continuing to do things the way they're already being done :) – warren Apr 1 '16 at 13:54
12

Benefits

  • Administration is efficient. Grading with large numbers of test takers can be automated either by using computer administration or by scannable answer sheets.
  • Once the scoring system is determined, there is no subjectivity in the application of the scoring system. This increases the reliability and predictive validity of the test. It also increases efficiency in that test takers are less likely to query the scoring.
  • There are tools to evaluate the internal properties of the test. These in turn allow for a principled assessment of reliability and also options for refining the test. Using classical test theory or item response theory, you have a range of tools to identify problematic items or response options. You can also evaluate the overall reliability of the test. Simple steps include looking at the proportion of people answering the item correctly (good items are approximately in the 50 to 70% correct region) and the correlation between answering the item correctly and the total score on the test (good items have item-total correlations above say .20 or so).
  • You can administer more items. A general principle of reliability of measurement is that if you can get more observations of behaviour, you will get more reliable measurement. If you write concise items, people can often do one item every minute or so. This is much more than what you get with open response items.
  • You can maintain comparability across alternate forms. There are tools particularly based on item response theory, which allow you to draw on a test bank. As long as you have a certain number of common items, you can change a test over time while still being able to compare scores from different versions of the test (e.g., to compare scores from year to year). Or alternatively, you may be concerned about test security, so different people get exposed to different items. Either way, it's easier to ensure equivalence of scores using multiple choice items.

Problems

Of course there are many problems with multiple choice tests.

  • They do a poor job of measuring certain skills and abilities. For example, they are poor at measuring written skills and interpersonal skills.
  • Writing a large number of good items takes time and skill. In particular, designing multiple choice questions that require deeper analysis rather than surface level recall is often challenging and requires some skill.

Concluding thoughts

Given all of the above, it is understandable why universities might use multiple choice tests to make admission decisions. For certain skills and abilities, they are an efficient means of getting a pretty good predictor of academic performance, especially compared to alternatives (e.g., grades that mean different things from past institutions).

That said, I'm aware of quite a few institutions at least in Australia that would combine a multiple choice section with a written component in an attempt to get a more complete picture of academic ability.

| improve this answer | |
2

Multiple question tests are easier to grade. In a lot of cases they also make the test easier to take and therefore push the number of passing students up a little so the instructor isn't failing too many students and doesn't have to use a massive curve.

| improve this answer | |
  • Yes simply this: It is cheaper. Unfortunately many institutions/professors are not really so much interested in quality as getting quotas filled. So they can use the time to do research which is what they really want to do. – joojaa Apr 1 '16 at 7:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.