5

Several recent questions on oral examinations have caused me to reflect on whether or not this particular examination tactic would be worth further investigation for my courses.

From a practical standpoint, I am stumped as to how one would go about grading an oral exam. For example, one of the key benefits of an oral exam from the student's perspective seems to be that the exam can be optimized in real time according to the student's performance; however, a well-prepared student may end up getting a relatively more challenging exam, while the ill-prepared student may receive a relatively easier exam.

All of this has me wondering:

How would one go about grading an oral exam in a manner which is both fair and accurate?

0

1 Answer 1

7

In all kinds of assessments, but especially oral exams, carefully prepared rubrics can help increase consistency and decrease subjectivity. From "A short guide to oral assessment":

One of the advantages of oral assessment is that it can often be marked quickly on the spot. To support this, the use of a marking guide or rubric of some sort is usually essential. The use of rubrics in oral assessment has many benefits:

  • It provides assessors with a common reference point for their judgments
  • It reduces the likelihood that judgments will be based on extraneous factors
  • Providing students with the marking guide in advance helps them understand the nature of good work and helps them to evaluate the quality of their own work in the assessment
  • It provides a basis for peer evaluation/feedback
  • It makes marking more efficient
  • It provides a useful framework for feedback to students.

Further details depend on the format of the exam, and on whether you want to include things like presentation skills in the grade or not.

In [1], there is a predetermined set of questions that are graded as follows:

Students are told that they will be asked a series of questions that are designed to evaluate their understanding of the material. Students are instructed to “think aloud” in order that their thought process can be observed (9). The grading system is explained and they are shown a scoring sheet. Four scores are possible for each answer: 3, 2, 1, or 0. Students answering a question correctly and without prompting earn 3 points. Each prompt a student receives results in a deduction of 1 point. Mistakes made along the way have no consequence on the grade if they are self-corrected. Following the questions in the order they appear on the scoring sheet allows for consistency in the administration of the examination.

In [2], the content of the exam varied with each student. Therefore,

Developing a rigid rubric for an oral examination was not feasible when each student individually determined the content. This required the evaluation criteria to remain roughly defined to facilitate evaluation of all students, no matter which reaction they chose. The rubric shown in Table 2 was used to guide the evaluators when grading. Presentation skills included oral and nonverbal communication (such as writing a reaction mechanism) and delivering information in an organized manner. Student knowledge about their chosen subject was evaluated partly through reviewing the content they had prepared for the session and whether they presented the required information. The ability of each student to answer questions was used to assess depth of knowledge and the extent of the research undertaken and to gauge how capably a student could synthesize information quickly, problem-solve, and think critically. For example, one student mentioned a recent innovation of their selected reaction being undertaken via solid-phase synthesis. Their knowledge regarding solid-phase techniques was subsequently assessed through further questioning to determine the depth of their understanding.

To limit inconsistencies and subjectivity, which are reported problems in oral testing, two of the course instructional staff were present for each examination

table 2

In the "poster exam" format described in [3],

The grading of the poster exam occurs in two parts. First students are graded on the content, organization, and design of the poster itself, the oral presentation of the poster, and their ability to answer questions on the poster. The students receive a group grade, which we chose to score as 60% of the exam. The detailed breakdown of the poster evaluation is shown in Figure 1.

Figure 1

...

Individual knowledge of the broad subject matter is probed in the second half of the test with an individual oral examination administered to each student in the poster group. Questions are drawn from a database constructed by the course designers and instructors, and labeled according to degree of difficulty. After each poster exam the database is refined and enlarged. The examiners are asked to draw questions randomly from the database and score the answers on a three-point scale of outstanding, acceptable, or unacceptable. Each student’s response is scored and entered on an answer sheet, which the students do not see, and returned to a folder that remains with the group. The next examiner then has access to the folder and does not repeat the prior questions asked. In this way each student receives at least two different oral exams, and most are given three. Examiners tend to conduct this aspect of the poster exam in one of two ways: some ask each student a different question, while others give the same question to all students.


[1] Roecker, L., 2007. Using Oral Examination as a technique to assess student understanding and teaching effectiveness. J. Chem. Educ, 84(10), p.1663.

[2] Dicks, A.P., Lautens, M., Koroluk, K.J. and Skonieczny, S., 2012. Undergraduate oral examinations in a university organic chemistry curriculum. Journal of Chemical Education, 89(12), pp.1506-1510.

[3] Marino, R., Clarkson, S., Mills, P.A., Sweeney, W.V. and DeMeo, S., 2000. Using poster sessions as an alternative to written examination—the poster exam. J. Chem. Educ, 77(9), p.1158.

4
  • 1
    "the use of a marking guide or rubric of some sort is usually essential": Mm... honestly, in 30 years of oral exams (student+teaching) I've never used such a rubric and I've never seen anyone using it. Jan 5, 2017 at 6:41
  • 2
    @Massimo Well, now you've met me :)
    – ff524
    Jan 5, 2017 at 6:42
  • We could do a research on the assessment uniformity with or without rubric ;-) Jan 5, 2017 at 6:46
  • Apologies for not accepting this answer sooner.
    – Mad Jack
    Feb 17, 2018 at 2:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .