In all kinds of assessments, but especially oral exams, carefully prepared rubrics can help increase consistency and decrease subjectivity. From "A short guide to oral assessment":
One of the advantages of oral assessment is that it can often be
marked quickly on the spot. To support this, the use of a marking
guide or rubric of some sort is usually essential. The use of rubrics
in oral assessment has many benefits:
- It provides assessors with a common reference point for their
judgments
- It reduces the likelihood that judgments will be based on
extraneous factors
- Providing students with the marking guide in advance helps them
understand the nature of good work and helps them to evaluate
the quality of their own work in the assessment
- It provides a basis for peer evaluation/feedback
- It makes marking more efficient
- It provides a useful framework for feedback to students.
Further details depend on the format of the exam, and on whether you want to include things like presentation skills in the grade or not.
In [1], there is a predetermined set of questions that are graded as follows:
Students are told that they will be asked a series of questions
that are designed to evaluate their understanding of the
material. Students are instructed to “think aloud” in order
that their thought process can be observed (9). The grading
system is explained and they are shown a scoring sheet. Four
scores are possible for each answer: 3, 2, 1, or 0. Students
answering a question correctly and without prompting earn
3 points. Each prompt a student receives results in a deduction
of 1 point. Mistakes made along the way have no consequence
on the grade if they are self-corrected. Following
the questions in the order they appear on the scoring sheet
allows for consistency in the administration of the examination.
In [2], the content of the exam varied with each student. Therefore,
Developing a rigid rubric for an oral
examination was not feasible when each student individually
determined the content. This required the evaluation criteria to
remain roughly defined to facilitate evaluation of all students,
no matter which reaction they chose. The rubric shown in
Table 2 was used to guide the evaluators when grading.
Presentation skills included oral and nonverbal communication
(such as writing a reaction mechanism) and delivering
information in an organized manner. Student knowledge
about their chosen subject was evaluated partly through
reviewing the content they had prepared for the session and
whether they presented the required information. The ability of
each student to answer questions was used to assess depth of
knowledge and the extent of the research undertaken and to
gauge how capably a student could synthesize information
quickly, problem-solve, and think critically. For example, one
student mentioned a recent innovation of their selected
reaction being undertaken via solid-phase synthesis. Their
knowledge regarding solid-phase techniques was subsequently
assessed through further questioning to determine the depth of
their understanding.
To limit inconsistencies and subjectivity, which are reported
problems in oral testing, two of the course instructional
staff were present for each examination
In the "poster exam" format described in [3],
The grading of the poster exam occurs in two parts. First
students are graded on the content, organization, and design
of the poster itself, the oral presentation of the poster, and
their ability to answer questions on the poster. The students
receive a group grade, which we chose to score as 60% of
the exam. The detailed breakdown of the poster evaluation
is shown in Figure 1.
...
Individual knowledge of the broad subject matter is
probed in the second half of the test with an individual oral
examination administered to each student in the poster group.
Questions are drawn from a database constructed by the
course designers and instructors, and labeled according to
degree of difficulty. After each poster exam the database is
refined and enlarged. The examiners are asked to draw questions
randomly from the database and score the answers on a
three-point scale of outstanding, acceptable, or unacceptable.
Each student’s response is scored and entered on an answer
sheet, which the students do not see, and returned to a folder
that remains with the group. The next examiner then has access
to the folder and does not repeat the prior questions asked.
In this way each student receives at least two different oral
exams, and most are given three. Examiners tend to conduct
this aspect of the poster exam in one of two ways: some ask
each student a different question, while others give the same
question to all students.
[1] Roecker, L., 2007. Using Oral Examination as a technique to assess
student understanding and teaching effectiveness. J. Chem. Educ, 84(10), p.1663.
[2] Dicks, A.P., Lautens, M., Koroluk, K.J. and Skonieczny, S., 2012. Undergraduate oral examinations in a university organic chemistry curriculum. Journal of Chemical Education, 89(12), pp.1506-1510.
[3] Marino, R., Clarkson, S., Mills, P.A., Sweeney, W.V. and DeMeo, S., 2000. Using poster sessions as an alternative to written examination—the poster exam. J. Chem. Educ, 77(9), p.1158.
