Diverse forms of testing and grading

Question

I am close to an academic who is facing heavy criticism for having used an unorthodox grading scheme in a final exams (there was a multiplication involved to ensure a good grade would mean decent success in two independent parts).

My main question is:

Q0: what are the main guiding principles you use when designing a test and a grading scheme? Are there any clear limitations imposed to you (legal, moral, or traditional; internal or external)?

Let us assume that the tests handed out are completely anonymous, so that it is given that the grading only depends on the test handed out, and on no other element. In particular, of course discrimination by gender or race or religious beliefs is not acceptable, but no need to mention it since (proper) anonymity should prevent it.

I would also like to gather a diverse array of ways to test and to grade. This type of "big list" question is sometimes seen at MathOverflow, I don't know if it is welcome here but I think seeing a variety of grading scheme and exam designs would help inform the answers to Q0.

Q1: What kind of exam design and grading scheme did you actually use (or enforce if you did not grade but coordinated graders)? What are the motivation behind this choice, and what did you conclude from the outcome? Multiple answers for various pairs of exam design/grading scheme is better unless they directly benefit from being gathered.

It is been asked in comment to specify, so let me add that I am more interested by scientific fields. The question clearly makes sense more generally, but is already quite broad.

Answer 1

I've designed a number of assessments, and also have had recent experience of a rather complex grade calculation going wrong for some students, so I would say first of all that it needs to be kept simple, so that it is transparent to students how you have arrived at the mark. Of course, there needs to be a close correspondence to the curriculum/learning objectives. We also try to think about applying Bloom's taxonomy, so that questions mix the descriptive and the evaluative.

• Q0: We have some constraints, some of them being mandated, but others really an attempt to standardise the rubrics within the department where I work. One of these, which makes a fair amount of sense, is to use an assessment grid, so that for each thing the student is graded on (e.g. level of critical analysis), there is an example of the kind of quality level expected within each grade boundary. We also have spent a lot of time thinking about group work and how to fairly grade it. A common approach for us now is to allocate an overall mark for a student group but also have an individual element, or allow for an adjustment based on individual performance. This is hard to get right, as you need to avoid a) team members not pulling their weight; but also b) one "stronger" team member taking all the work on themselves.

• Q1 These days I avoid exams in the main, and use mostly assessed presentations, vivas and poster presentations. The grading scheme typically is something like 70/30 aspects of between content and presentation. As I teach HCI & Information Science, there often needs to be evidence of following a structured design process, rather than merely giving this process lip service. This is almost always clear to see in the student work if it has been done well / at all.

Answer 2

My experience comes from pure math in the US, with no clear limitations. My approach depends on the course, and I usually try things a little differently each semester, but my main design principles are:

• first, determine what are the main things I think a student should definitely be able to do upon finishing the course (e.g, for differential calculus: compute limits and derivatives, understand the relation with tangent lines, simple optimization problems), as well as things I would like students to be able to do (e.g., more complicated optimization problems, logarithmic differentiation, conceptual mastery, ...). this gives me a set of benchmarks for the students to both get passing grades (meet minimum requirements), as well higher-tiered benchmarks to get B's or A's.

• try to make an exam that tests as many of these things, as independently as possible, and in varying level of difficulty, with a constraint that a very good student should be able to finish in about half the allotted time

• try to make questions that are straightforward to grade, both for ease of my grading as well as to improve uniformity of grading

• try to use problems similar to what they have seen before on homework/previous exams/in-class examples, but for the most part slightly different,

As for some ways I (attempt to) put this into practice (say for calculus):

• make true/false to test conceptual understanding (effect: most students get very close to the same number of true/false right, so this has little effect in differentiating grades except for a few students who do very well or very badly)

• for the other problems problems (some of which the students may have to show work for, and some of which I allow them to omit work if they wish), essentially the grading scheme is: full credit (very minor errors allowed), half credit, or no credit (effect: grading is easy, and I think rather fair)

• what I need to give out at the end are letter grades (A, B, C, D, F); based on the exam questions and the benchmarks I have in mind, I set cut-offs for the number of problems a student I think should get right to get, say an A or a C, and convert this into a number of points. (in practice: I often find I there are too many low grades, and end up looking at individual final exams to reevaluate what I think various students deserve, and revise my cut-offs after grading)