23

TL;DR:

How should I blindly grade exams when I can recognize over half of the papers by the handwriting?

Situation: I'm a TA for an first year physics class of approximately 50-60 students. Due to a complaint along the lines of "the grader hates me so I failed the class" that the department got last year, there is a proposal in the department to ensure that tests be swapped to blind grading. However, as the class assigns a fairly large amount of homework, from experience I know that I can reliably recognize more than half of the class's handwriting. The proposed system goes along the lines of having students use an ID number that they get when they take the test that isn't shown to me until I'm putting grades into the computer. The problem is that I feel that this would do very little to actually reduce bias if TAs can just recognize handwriting without a name.

Short of putting people on rotation for grading this stuff, which has its own problems, are there any better ways to implement a blind grading system to remove any handwriting bias?

8
  • 15
    As the TA this is not your issue, but: I find the proposal to switch to a blind grading system because the graders cannot be trusted to grade fairly problematic. It seems to -- albeit in a weak, tacit way -- endorse the possibility that the graders are unfairly biased against individual students. But that is a serious enough charge that it should be affirmed or denied. If graders can show that they are in fact grading consistently according to a clear standard, case closed. On the other hand, if your personnel are in fact untrustworthy you should find out: this has other implications... Jul 17, 2014 at 20:02
  • 19
    @PeteL.Clark: I disagree strongly. We started grading blindly because we know we are less ojective if we know the author. You don't have to have a grudge; simply knowing the students approximate skill changes how you read their answers, and then comes personal interaction. Blind grading removes all that from the equation -- but for handwriting. (I rarely recognise handwriting. Even if I can place one, I'd probably only know "one of these four", not more.)
    – Raphael
    Jul 17, 2014 at 20:43
  • 1
    @Raphael: First, who are "we"? Second: yes, there are issues with enforcing consistency and correctness of grading. But grading "blind" only addresses these indirectly: you can still be biased against an author or grade inconsistently and inaccurately if you don't know who the author is. (For instance, if someone demonstrates a misunderstanding on one problem, when you grade all the subsequent problems you have that misunderstanding in mind.) The way to compensate for lack of objectivity is to adopt a grading style that forces you to keep your objective goals in mind and follow them. Jul 17, 2014 at 20:49
  • 1
    @PeteL.Clark 1) My group. We give undergrad algorithms lectures. 2) That is true. We try to mitigate by grading by problem, not by hand-in (this has other advantages) and agree on reasonable precise grading schemes for each problem. We verify the consistency of its application by making a second pass with doctorands only (which don't see any student handwriting during the semester, too). Of course, you still need to want to stay objective. Personally, I have found the effect of having no names on the exams makes this a lot easier as it enables the other measure I mention to be effective.
    – Raphael
    Jul 17, 2014 at 20:53
  • 4
    Also, think about this as a reaction to the specific allegation "The grader hates me so I failed the class." Which is better: "No, don't worry: the grader doesn't know who you are [unless of course she figured out who you are, which everyone seems to agree happens at least some of the time, but we hope it doesn't happen too often]." Or "How the grader feels about you is not relevant. Either the work has been graded correctly according to this explicit standard or not. Let's determine whether that has occurred." Jul 17, 2014 at 20:56

5 Answers 5

20

If I understand this correctly, blind grading is a formality introduced by the University to make students' claims on the biased teachers unjustified. I would say - keep it that way and do not bother any more. You are not going to hate them and give them lower grades, are you? So, it does not really matter that you in fact can recognise the handwriting. If a student is really really bothered to keep their identity secret... well, they could try to write with the unusual hand.

Answering your question formally - to implement a real blind grading system you can swap the scripts between departments, or TAs.

3
  • 12
    ...or require all homework to be typeset, so you can't learn students' handwriting.
    – JeffE
    Jul 17, 2014 at 18:39
  • 8
    @JeffE There's still an awful lot you can tell from the students' typesetting. When I graded Computer Science assignments, there was a big difference between the MS Word writeups, the inexperienced LaTeX writeups, and the experiences LaTeX writeups. Things like vocabulary choice, common typos, etc., can still make it easy to recognize students. Even if you don't know their names, it'd be relatively easy to recognize "that's student 7892's assignment, and this is student 2034's." Jul 17, 2014 at 21:50
  • 8
    @JoshuaTaylor I think Jeffs point is that, if the homeworks are typeset, you can't recognise anything in the handwritten exams. Jul 17, 2014 at 23:18
10

Others have mentioned the issue of removing hand-writing from the equation by having homework or assignments printed as opposed hand-written. so I will simply add what we commonly do in my department.

Add a moderator to a random sample of scripts/exams.

With a moderator, even if the grader knows the student, the grader must consider that their own marking will be checked. If Gary Grader consistently mis-grades papers (due to bias or simply weak grading skills) then Mary Moderator should catch this, at least on some. Those flags may indicate a need for someone to step in and talk with the grader about grading bias or other grading issues.

Now, you could make the moderating blind as well so that Mary Moderator would not know which grader she was moderating. One reason to do this is that Mary might be concerned that George will be angry at her (or take revenge, etc.) so Mary might simple say that "Yes, all of Gary's papers are graded correctly."

We don't do blind moderation but I have heard others comment that "we don't change grades because we don't want to cause problems for our colleagues" which disturbs me greatly (because the graders know this and it, in effect, nullifies the whole moderation process).

8
  • +1 for moderation. Also add a marking scheme, in my experience it works well in CS and maths, don't see why not use it in physics. If I was in the OP's situation, I wouldn't bother about recognizing the handwriting, I would rather be concerned that the university management treats me as a potentially dishonest employee. If there were complaints from the students, they should be investigated and discussed by the relevant university board instead of tarring all the lecturers as unable to provide unbiased assessment. Jul 18, 2014 at 9:37
  • Taken further, in the UK all exams are graded by at least three professors (and one of them from another university), and the marks are reviewed by a committee if they differ a lot. This is a slow process, but students can be sure they have been graded fairly.
    – Davidmh
    Jul 18, 2014 at 10:48
  • @Davidmh Looks like you have a very interesting data set to run statistics on. Do you run some analysis? What is, for instance, the variance among grades given by different professors to the same exam? Jul 18, 2014 at 10:54
  • @FedericoPoloni I don't. I just have friends that have studied in the UK. IIRC, the committee reviews the grading if the difference is above 5-10%, so they want to be quite accurate.
    – Davidmh
    Jul 18, 2014 at 11:09
  • @Davidmh I do not believe that ALL exams in the UK are graded the way you describe. The one where I teach simply has a single grader and the module leader will moderate.
    – earthling
    Jul 20, 2014 at 11:31
2

I'm not entirely sure what you mean by rotating (and the associated problems), but we use the following scheme:

  • Graders have "their" questions.
  • exams "rotate" between the graders until in the end each question is corrected for each exam.
    • Sometimes this is done in large batches - each grader goes to the secretary when they have time, get all or maybe half of the exams, vanishes to their office, and returns exams and marks list when done.
    • More often, all graders meet, and small batches of exams are exchanged. Here, everyone gets one question, and as soon as you are finished (or cannot easily find a new batch) you either get a new question assigned or towards the end help sort who still needs to get which question and carry those exams where they need to go. This way, the exams are quite well shuffled, so one would expect that even if a drift in the marking over the course of going over all exams occurs, this is different for each question and cancels out in the end.

There are two main ideas with this scheme:

  • Even if there are example solutions and a rough list of points the professor wants to be considered, usually a bunch of minor decisions has to be made. It is easier to have consistent marks for all students if the marking for a given question is done by one grader.
  • It is faster if every grader has to go into the details only for a few questions. Getting "into" a question can take considerable time, and after solving the question myself and comparing that to the example solution I tentatively write up criteria for the marking. I then grab some 10 - 20 exams and go through the question marking them but without actually giving grades and check whether my tentative point list is feasible and what other typical problems I need to add to the list. Only then the actual grading starts.

I cannot recall whether we have the students pseudonymized (by student ID) or whether we have names - typically the pressure is to get the grading done, and that means I (and about all of my colleagues but the one who transfers grades to the final result list) never look at the name. I directly go to the sheet where "my" question is (or sometimes, particularly towards the end, people who are already finished will do that for others).

  • With such a scheme, the impact a single grader biased against a particular student can have is quite limited.
    BTW: some of us usually "know" the students from labwork practicum (others help only with the exams and have never seen any of the students) - but that are brief encounters of large numbers of students. A student has to be either exceptionally good or exceptionally bad or behaving exceptionally in some way to be remembered by name. At least for me, while I may remember someone when seeing them, the connection to name or handwriting is much weaker. Of course, that is different if you encounter them again and again over the duration of the course.

I'd like to add some general thoughts:

  • While I do see distinct advantages with the system described above which can easily be carried out in a blinded fashion, the student claim

     My grader hates me so i failed the class

    is not among the good reasons. Students can look at their marked exams. They can then complain if they suspect the grading was not fair (or an error occured). They can even ask for someone else grading their exam. If it turned out that there was a problem with the grading, this is dealt with a) for the student (or all students in the each-grader-has-their-questions scheme) and b) will have consequences for the grader.

  • In my experience, students thinking that TAs not only remember them but have strong enough feelings that they end up giving biased marks to a measurable extent (in a field where a large part of the grading scheme is usually formulated in "hard" ways like calculation correct yes/no; sign error, axis label missing, units missing: each -1/2 point and so on) greatly overestimates the importance the given student has to the typical grader (PhD students or postdocs who are assigned to help grading on top of all other work they have)

  • Do not forget that there is a distinct bias against letting students fail: failing students come again for examination and create lots of additional, but unsatisfactory (IMHO it is far more pleasant to correct a good exam than to sort out the mess of a bad exam) work.

IMHO there are some trade-offs involved. Obviousy, possible bias against students is bad and needs to be avoided. On the other hand, you have to be careful not to make otherwise bad decisions and end up unbiased, but overall worse:

  • One obvious way of not having a student bias (that also removes the information-leaking of the question) is to have graders that were not otherwise involved with those students.
    OTOH, people who have been TA with these students may have a far better grasp what can be expected from the students than people who have not been involved with that course (we have a fair amount of PhD students and post-docs helping with the grading that have a related but not the same background, e.g. physicists and optical engineers helping grading physical chemistry exams).

  • Another obvious possible bias is that seeing what the other grader gave for their questions will influence the grader*. This could be removed by having separate sheets for each question - at the cost that it is error prone and/or a whole lot of additional work to make really sure nothing gets lost, nor mixed up.

  • Already having only student ID or other pseudonyms in practice means that some easy way of error checking is removed: it is much easier to mix up the grades between students when transfering to the grade list when the list consists of longish ID numbers instead of names.
    Obviously there should not be any errors, but such errors do occur. And at some point the question should be asked whether the removed alleged bias is worth that less time can be spent on actually teaching the students because it is spent on double checking anonymized lists and correcting errors that occured.

* (But for our exams it is rather typical that students perform very differently in different subfields: one is at home with thermodynamics but doesn't get kinetics and vice versa)

1

The blind grading should never allow the name of the student to be linked with the student number. Your grade sheets should be coded only by student number until the end of the semester. All written assignments, wether marked or unmarked, should be identified by number. In this way, you will have much less of a chance to learn the handwriting. You will get the odd look when a student shows you a paper to ask a question.

6
  • 3
    I'm not sure that such thorough anonymity is practical in many course settings. If the OP were merely the grader, I could see this working well, but he said that he is the TA. So presumably he is also interacting with the students in person and fielding questions about their problem sets, exams, and the grading thereof. In many contexts, making sure that you never see a student's handwriting would make it difficult or impossible to fulfill the responsibilities of a TA. Jul 17, 2014 at 19:50
  • 3
    @PeteL.Clark And it's not just about handwriting. If a student comes to you asking about a misunderstanding, it can be be pretty obvious when that same misunderstanding appears in an assignment. Even without face to face interaction, other features make homeworks identifiable through the course of the semester. It might be impossible to put a name to students after two assignments, but it's often not too hard to recognize that submission78 for homework1 and submission 93 for homework2 came from the same student. Jul 17, 2014 at 21:54
  • 1
    @Joshua: Yes, I agree. Upshot: the measures necessary to seriously enforce anonymity seem in many cases to be antithetical to the TA job. Jul 17, 2014 at 22:18
  • @PeteL.Clark I agree entirely; I don't think that anonymity is all that it's cracked up to be. It's awful hard to help students if one of the typical ways of recognizing their difficulties is removed. Jul 17, 2014 at 22:23
  • The first time a studnet asks for a regrade, you can attach that student to their ID number.
    – JeffE
    May 12, 2016 at 11:52
1

To help "save me from myself," from the first moment I was allowed to implement partial credit (after escaping my alma mater's absolute insistence on all-or-nothing grading), I began crafting incredibly, at times ludicrously detailed rubrics (down to the half-point, even for the many questions on a 100-point exam) detailing every conceivable level (and timeline) of errors that a student could make (or at least, that I could anticipate them making), and I make myself stick to it. Even if the student has annoyed me with his in-class noisy chattiness--even if he has already been caught in an academic misconduct net earlier in our acquaintance--it forces me to treat him the same as every other student. If it's a 3-point mistake for someone else, it's a 3-point mistake for him, too.

(Sidebar, Your Honor: Sorry, fellas, for the gender bias there, but after a single XX to start the Sad Roll of Dishonor, it's been an unbroken streak of about 76 in a row for the XY's.)

I also grade each exam one page at a time, to help break up the exams and make the handwriting-recognition neurons fire less often. That also has the fringe benefit of keeping me from turning any concerns that might be forming about trends in the grades so far (since I don't know the overall results yet) into an inappropriate shifting of the standards bar halfway through.

2
  • Something I used to do (I no longer teach) was to grade in such a way that there should not be any problems if any or all students compared their graded tests with any or all other students in the class, and this certainly happened in my classes (and I did it myself sometimes as a student). Another thing I started doing (after a few years of teaching) was to photocopy my solutions/rubric sheet and handing a copy of it back with each test. Even before this I often handed solutions out to save class time (and office hour time), and at some point I realized I could save even more (continued) May 12, 2016 at 15:46
  • (continuation) of my personal time by simply handing back what I had already hand-written for grading purposes, and not try to rewrite it again in a neater form. The solutions were often brief, but for rubric purposes I would always show steps I expected some students to miss, so it actually worked fine with most students (who could, and often did, ask their neighbor for clarification anyway). As for the rubric, this was often a "work in progress" in the sense that I made grading decisions whenever a certain type of error showed up, rather than trying to anticipate them in advance. May 12, 2016 at 15:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .