Blind grading for exams isn't very blind

Question

TL;DR:

How should I blindly grade exams when I can recognize over half of the papers by the handwriting?

Situation: I'm a TA for an first year physics class of approximately 50-60 students. Due to a complaint along the lines of "the grader hates me so I failed the class" that the department got last year, there is a proposal in the department to ensure that tests be swapped to blind grading. However, as the class assigns a fairly large amount of homework, from experience I know that I can reliably recognize more than half of the class's handwriting. The proposed system goes along the lines of having students use an ID number that they get when they take the test that isn't shown to me until I'm putting grades into the computer. The problem is that I feel that this would do very little to actually reduce bias if TAs can just recognize handwriting without a name.

Short of putting people on rotation for grading this stuff, which has its own problems, are there any better ways to implement a blind grading system to remove any handwriting bias?

Answer 1

If I understand this correctly, blind grading is a formality introduced by the University to make students' claims on the biased teachers unjustified. I would say - keep it that way and do not bother any more. You are not going to hate them and give them lower grades, are you? So, it does not really matter that you in fact can recognise the handwriting. If a student is really really bothered to keep their identity secret... well, they could try to write with the unusual hand.

Answering your question formally - to implement a real blind grading system you can swap the scripts between departments, or TAs.

Answer 2

Others have mentioned the issue of removing hand-writing from the equation by having homework or assignments printed as opposed hand-written. so I will simply add what we commonly do in my department.

Add a moderator to a random sample of scripts/exams.

With a moderator, even if the grader knows the student, the grader must consider that their own marking will be checked. If Gary Grader consistently mis-grades papers (due to bias or simply weak grading skills) then Mary Moderator should catch this, at least on some. Those flags may indicate a need for someone to step in and talk with the grader about grading bias or other grading issues.

Now, you could make the moderating blind as well so that Mary Moderator would not know which grader she was moderating. One reason to do this is that Mary might be concerned that George will be angry at her (or take revenge, etc.) so Mary might simple say that "Yes, all of Gary's papers are graded correctly."

We don't do blind moderation but I have heard others comment that "we don't change grades because we don't want to cause problems for our colleagues" which disturbs me greatly (because the graders know this and it, in effect, nullifies the whole moderation process).

Answer 3

I'm not entirely sure what you mean by rotating (and the associated problems), but we use the following scheme:

• Graders have "their" questions.
• exams "rotate" between the graders until in the end each question is corrected for each exam.
  • Sometimes this is done in large batches - each grader goes to the secretary when they have time, get all or maybe half of the exams, vanishes to their office, and returns exams and marks list when done.
  • More often, all graders meet, and small batches of exams are exchanged. Here, everyone gets one question, and as soon as you are finished (or cannot easily find a new batch) you either get a new question assigned or towards the end help sort who still needs to get which question and carry those exams where they need to go. This way, the exams are quite well shuffled, so one would expect that even if a drift in the marking over the course of going over all exams occurs, this is different for each question and cancels out in the end.

There are two main ideas with this scheme:

• Even if there are example solutions and a rough list of points the professor wants to be considered, usually a bunch of minor decisions has to be made. It is easier to have consistent marks for all students if the marking for a given question is done by one grader.
• It is faster if every grader has to go into the details only for a few questions. Getting "into" a question can take considerable time, and after solving the question myself and comparing that to the example solution I tentatively write up criteria for the marking. I then grab some 10 - 20 exams and go through the question marking them but without actually giving grades and check whether my tentative point list is feasible and what other typical problems I need to add to the list. Only then the actual grading starts.

I cannot recall whether we have the students pseudonymized (by student ID) or whether we have names - typically the pressure is to get the grading done, and that means I (and about all of my colleagues but the one who transfers grades to the final result list) never look at the name. I directly go to the sheet where "my" question is (or sometimes, particularly towards the end, people who are already finished will do that for others).

• With such a scheme, the impact a single grader biased against a particular student can have is quite limited.
  BTW: some of us usually "know" the students from labwork practicum (others help only with the exams and have never seen any of the students) - but that are brief encounters of large numbers of students. A student has to be either exceptionally good or exceptionally bad or behaving exceptionally in some way to be remembered by name. At least for me, while I may remember someone when seeing them, the connection to name or handwriting is much weaker. Of course, that is different if you encounter them again and again over the duration of the course.

---

I'd like to add some general thoughts:

• While I do see distinct advantages with the system described above which can easily be carried out in a blinded fashion, the student claim

   My grader hates me so i failed the class

  is not among the good reasons. Students can look at their marked exams. They can then complain if they suspect the grading was not fair (or an error occured). They can even ask for someone else grading their exam. If it turned out that there was a problem with the grading, this is dealt with a) for the student (or all students in the each-grader-has-their-questions scheme) and b) will have consequences for the grader.

• In my experience, students thinking that TAs not only remember them but have strong enough feelings that they end up giving biased marks to a measurable extent (in a field where a large part of the grading scheme is usually formulated in "hard" ways like calculation correct yes/no; sign error, axis label missing, units missing: each -1/2 point and so on) greatly overestimates the importance the given student has to the typical grader (PhD students or postdocs who are assigned to help grading on top of all other work they have)

• Do not forget that there is a distinct bias against letting students fail: failing students come again for examination and create lots of additional, but unsatisfactory (IMHO it is far more pleasant to correct a good exam than to sort out the mess of a bad exam) work.

IMHO there are some trade-offs involved. Obviousy, possible bias against students is bad and needs to be avoided. On the other hand, you have to be careful not to make otherwise bad decisions and end up unbiased, but overall worse:

• One obvious way of not having a student bias (that also removes the information-leaking of the question) is to have graders that were not otherwise involved with those students.
  OTOH, people who have been TA with these students may have a far better grasp what can be expected from the students than people who have not been involved with that course (we have a fair amount of PhD students and post-docs helping with the grading that have a related but not the same background, e.g. physicists and optical engineers helping grading physical chemistry exams).

• Another obvious possible bias is that seeing what the other grader gave for their questions will influence the grader*. This could be removed by having separate sheets for each question - at the cost that it is error prone and/or a whole lot of additional work to make really sure nothing gets lost, nor mixed up.

• Already having only student ID or other pseudonyms in practice means that some easy way of error checking is removed: it is much easier to mix up the grades between students when transfering to the grade list when the list consists of longish ID numbers instead of names.
  Obviously there should not be any errors, but such errors do occur. And at some point the question should be asked whether the removed alleged bias is worth that less time can be spent on actually teaching the students because it is spent on double checking anonymized lists and correcting errors that occured.

* (But for our exams it is rather typical that students perform very differently in different subfields: one is at home with thermodynamics but doesn't get kinetics and vice versa)

Answer 4

The blind grading should never allow the name of the student to be linked with the student number. Your grade sheets should be coded only by student number until the end of the semester. All written assignments, wether marked or unmarked, should be identified by number. In this way, you will have much less of a chance to learn the handwriting. You will get the odd look when a student shows you a paper to ask a question.

Answer 5

To help "save me from myself," from the first moment I was allowed to implement partial credit (after escaping my alma mater's absolute insistence on all-or-nothing grading), I began crafting incredibly, at times ludicrously detailed rubrics (down to the half-point, even for the many questions on a 100-point exam) detailing every conceivable level (and timeline) of errors that a student could make (or at least, that I could anticipate them making), and I make myself stick to it. Even if the student has annoyed me with his in-class noisy chattiness--even if he has already been caught in an academic misconduct net earlier in our acquaintance--it forces me to treat him the same as every other student. If it's a 3-point mistake for someone else, it's a 3-point mistake for him, too.

(Sidebar, Your Honor: Sorry, fellas, for the gender bias there, but after a single XX to start the Sad Roll of Dishonor, it's been an unbroken streak of about 76 in a row for the XY's.)

I also grade each exam one page at a time, to help break up the exams and make the handwriting-recognition neurons fire less often. That also has the fringe benefit of keeping me from turning any concerns that might be forming about trends in the grades so far (since I don't know the overall results yet) into an inappropriate shifting of the standards bar halfway through.