36

I'm trying a new type of essay in the community college class that I teach. Once a week the students write a 700 word analysis of a topic I choose. It's worth a regular essay (a measly 10 points) but I've been trying a new grading method.

Turn in the paper (1 points)

Peer analysis x 3 (3 points each)

In the peer analysis, students dissect three of their classmate's essays and give them between 0 and 3 points. On average, I find that students give a higher grade than I would.

The main issue I'm facing is that 19 of my 30 students aren't even bothering to do the peer review. By default, they are giving their peers F's on their assignments. This is vastly dropping my class's average score, and as a relatively new teacher, I'm somewhat concerned that I'll be penalized for their inability to do their work. In my opinion, bumping up everyone's grades would be immoral.

  • 85
    Why are you expecting the students to do the grading, rather than doing it yourself? Are they all being paid as TAs? – Patricia Shanahan Oct 26 '16 at 1:38
  • 44
    As an aside, doing this in a highly competitive environment is also a questionable idea. Grading from peers will inevitably lead to resentment and (probably justified) accusations of unfairness by some graders. – user8001 Oct 26 '16 at 2:19
  • 113
    "I've been trying a new grading method": Now that you've tried it, I suggest you to go back to the good ol' method before soon. – Massimo Ortolano Oct 26 '16 at 2:37
  • 40
    Imagine yourself in the shoes of a student whose grade suffers because some other student blew off their work. Does this really strike you as fair? – user37208 Oct 26 '16 at 5:11
  • 63
    "By default, they are giving their peers F's on their assignments." --- this is where I have a big problem. Not doing the work should be an automatic F for the grader, and the person whose work should have been graded now gets graded by you. They should be unaffected. – RemcoGerlich Oct 26 '16 at 9:55
178

You seem to have the grading scheme mis-targeted.

When you make peer review part of your grading scheme you grade the review not the paper reviewed. Failure to do the review should be reflected in the reviewer's grade not in the author's grade.

The criteria I use for grading reviews are along the lines of

  • Does the review correctly identify strengths and weakness of the submitted material?
  • Does it communicate problems clearly and helpfully to the author?
  • Are the criticisms couched in constructive language? (I know, this is not actually necessary or honored in the real world, but I want my classroom to be a comfortable learning environment so I insist.)
  • Does the review identify what parts of the paper are the source of particular complaints. ("Section 3.2 is unclear because..." as opposed to "Some parts of the paper are unclear").

Note that this means I have to read and review the paper myself and then read and grade the reviews. It's painfully time-consuming.

The positive aspects of this kind of assignment is that it engages the students at the "evaluation" tier of Bloom's taxonomy, and exposes them to a range of different quality of work so they get a chance to understand for themselves that some writing is just plain better than other samples.

The negative aspects include the amount of time used, the somewhat subjective nature of the grading criteria, and a tendency for students to tell each other how very well they are all doing (it requires a certain expertise to do a good—and especially a critical—job of evaluating the work of others; if your students aren't at that level yet they won't set high standards for their peers).


Addendum

Over the weekend I saw a talk on a system called calibrated peer review, which provides a framework for using peer review as part of the grading scheme for student essays. I have no personal experience with the system.

The whole system is mediated over the internet. Assignments proceed in several phases.

  1. Student receive the prompt, do any necessary research, then write and submit their response.

  2. Students then receive a rubric for assessing responses to the same assignment and apply it to three sample responses provided by the instructor (designed to mimic poor, adequate and good student responses).

    Success in this stage is defined by having a rating close to that assigned by the instructor. They are allowed a second chance on any where they deviate excessively.

    The weight of a students review in the third stage is determined by their performance in this stage (students whose use of the rubric does not match the instructor's will have less impact then student whose application of the rubric matches the instructor's).

  3. Students then read and rate three anonymous responses from their peers (and possibly from the instructor who can provide a 'student' response as well as the three used for calibration). These responses are factored into the author's grade on the assignment using the weights established in stage 2.

    Finally, students read and rate their own response. Success here defined by matching the consensus.

    Disputes and special cases are resolved by the instructor.

Clearly there is a lot going on there. The anonymity, the use of the calibration stage, and the weighting of student evaluations by their success in rating the calibration responses all work to prevent the worst failure modes of just asking students to grade each other's work directly. Even so, you need that rule that the instructor can overrule the system in exceptional case.

When all is said and done a student's grade on the assignment can be based more on their rating (of samples, of their peers's responses, and of their own submission) than on their own writing. The speaker showed a grading scheme he used in which the essay itself was only worth 20% of the assignment grade with the rest coming from various parts of the review sequence.

This is a lengthy process and is used more on short writing assignment than on substantial papers. It also requires considerable prep, as the instructor must provide the prompt, the rubric, three calibration responses and possibly a fake 'student' response.

  • 9
    This is a really nice teaching method. (+1) – henning Oct 26 '16 at 8:58
  • 5
    Assuming we are talking about undergrad & up students (maybe high school too), the ability to do a fundamented critical analysis is more important than just soak up and parrot knowledge. The method above is really nice, and I had some professors use something like that on a course I took. Challenging but rewarding. – Mindwin Oct 26 '16 at 11:48
  • 4
    +1 as @RemcoGerlich said above, "Not doing the work should be an automatic F for the grader," not the gradee. – MissMonicaE Oct 26 '16 at 14:04
  • It's actually quite simple: grade should represent the quality of work. If no evidence is available, best guess on work's quality would be F only for students that are already failing mercilessly. Now, one could keep the grade on a safe (lower) side of confidence interval - or even outright penalize not turning papers in - to create incentive for providing sufficient evidence for grading. However, that's a different mechanism and it has gone wrong in OP's case bacause of misplaced incentives. – Daerdemandt Oct 28 '16 at 8:01
  • The thing is, it's not clear OP's course is supposed to evaluate the student's skills as reviewers. While it's an important skill - should s/he really have students practice it in this specific course? – einpoklum Oct 30 '16 at 14:00
97

If I understand right, you are penalizing students who did nothing wrong, because others were supposed to peer review their work and didn't? And you didn't bother to grade your own students' essays? Nothing personal, but if I were your department chair I would see to it that this came to an immediate end.

Students' grade should be based on their own work, and you should be the one to evaluate it. To outsource this work to your students is to abrogate your responsibility as instructor.

Peer review can be a great thing. If you want students to review each others' work, and to hold students accountable for writing thoughtful reviews (or at least for writing something), then grade students on their own work, and also on the quality of the reviews that they write about others' work.

  • 28
    I think the essential part here is the last paragraph. There is nothing wrong with having the students "peer review" but if this is supposed to make any sense you have to grade the "peer reviews" AND the original assignments. – DRF Oct 26 '16 at 9:06
  • 1
    @DRF But after having graded the "peer reviews" he could take the results of those "peer reviews" that he graded as good to help him grade the original assignments with maybe even gaining statistical accuracy. This way he could kind of outsource some work. The net effect might be positive but it depends on the average quality of "peer reviews". By giving points for good "peer reviews" he might incentivize high quality "peer reviews". Not sure I would like this outsourcing. – Trilarion Oct 27 '16 at 8:37
11

The following two sentences demonstrate a severe misunderstanding of what's happening:

By default, they are giving their peers F's on their assignments.

And:

This is vastly dropping my class's average score, and as a relatively new teacher, I'm somewhat concerned that I'll be penalized for their inability to do their work...

The first sentence is written such that students give each other grades. The second sentence is written such that you give your students grades. The second sentence is correct (or at least it is in every country I've worked in).

This is not to say you cannot assign students to give each other feedback or to provide peer assessment. Instead, the point is that this Q's peer assessment is not X's student's grade. You can make:

  1. Q's completion of the peer assessment assignment a part of Q's grade
  2. Q's assessment a (small) component in X's grade

But clearly in the case where Q fails to complete this component of their work for the class, X cannot be penalized.

It may legitimately be lowering the grades of the 19 students who are not doing it, but then that's a different problem (one of motivation or clarity of assignments or level vs. student ability) rather than the problem that seems described in the OP.

6

Giving your students first experience with peer review is a good idea. You can grade them on their ability to do peer reviews. Maybe, a small fraction of their grade can also come from how their peers graded them, but never 90%.

Right now, underachieving students are probably unfairly advantaged as you pointed out. At the same time, the best students are probably not getting their due credit. Only average students would remain unaffected, except that they can also randomly suffer when their reviewer simply doesn't review them. For what reason do you lay the reviewer's failure onto the reviewed student?

  • 1
    "Maybe, a small fraction of their grade can also come from how their peers graded them, but never 90%." - I see a general problem in that the students are grading something which they are just learning how to do at the same time. Note that each student has to write the essay, and each student will write that essay the way they think is correct. And at the same time, without even having received any feedback on what was indeed good or bad about their way of writing the essay, they are supposed to also judge other essays. I think the only way this can work is to split writing and reviewing ... – O. R. Mapper Oct 26 '16 at 7:33
  • 1
    ... phases up to consecutive weeks, and - if the review result is indeed supposed to influence the other students' grades - maybe reserve the actual grading part for the very last assignment during the course/unit, when the students in the class can be reasonably considered "knowledgeable" about that type of essay. – O. R. Mapper Oct 26 '16 at 7:36
  • 1
    It's not necessarily a good idea. It very much depends on the course. Students should not get a first experience of too many things at once. – einpoklum Oct 30 '16 at 14:01
3

I think you miss the way to force students to DO the grading. I thought of simillar peer grading, but with rules:

  1. Student will grade their classmate essay/report according grading rules and mark errors. (0 - 10 pts.)
  2. I will grade the essay according the very same grading rules. (0 - 10 pts.)
  3. I will grade the peer grade (-inf - 10 pts.) 10 points default, -1 for false positive (they miss an error) and -1 for false negative (they mark correct part as an error).
  4. Students score is given by weighted sum of peer grade (#1), my grade (#2) and peer quality (#3).

    Applying Rule #3 you can mark the default-graders with F without any chance to complain. You can find as many false negatives/positives as you want to ruin the score of their essay :)

Be careful with grading false positives though. Scrap the points for errors they actually can recognize as errors (they were covered in the lecture; they were mentioned in the grading rules).

  • -1 for the "You can find as many false negatives/positives as you want to ruin the score of their essay :)" - Grading should be fair and objective, there should be no need to "ruin the score" of anyone – user2813274 Oct 30 '16 at 20:53
  • @user2813274: When somebody fails to grade fairly, and default F or default A is unfair grade, then they does not deserve to pass, do they? Simillar result but with different wording can be: To pass, the student must get at least X points from each task. But it is vulnereable to Default-F schoolmate. – Crowley Oct 31 '16 at 9:20
2

I suspect part of the reason you're even trying this out is the lack of budget for teaching assistents, or even for paying you personally to check those essays. If that's the case - it's really too bad, and I would consider trying to do something about that (perhaps through some kind of collective action by teachers? Through a union maybe?).

Also, I would suggest that if you're trying out a new pedagogical mechanism you would start your trial by a discussion with students at the beginning of the semester and the reaching of a tentative understanding, if not proper concensus, regarding how the experimental grading mechanism should work. That's at the very least. I believe this would either improve student participation or at least inform you why it is unlikely to be as high as you would prefer.

Finally - I object to your excessive focus on grades. What's important is that students get comments, suggestions, pointing-out-of-errors - on their work. That's the qualitative feedback they can and should learn from, not the point on some scale on which you put them. If that were the case, it would be meaningless to "give an F" on an essay, and reviewing could only result in something substantive.

0

Just because I think some of the answers here are misinterpreting the question here is my view.

You are saying that each student rates the other students papers and writes a brief opinion regarding it, and that each rating gives the rater 3 points, right? That is a great exercise and it definitely encourages support between students!

However, you are saying that 18 of the students won't do the peer review portion of the exercise, right? Well, if that is the case and they do not have any legitimate reason to ignore the assignment (protesting the subject you gave them as not reasonable, thinking you are outsourcing grading, etc) then you should just not give them the points as a reviewer. They failed to do it. However, if they do have a legitimate reason you should still take the points away. If they have an issue then they should speak to their advisor or someone with the department. Ultimately, the professor has the final say on what happens in a class, not the students, and while yes... there are things you shouldn't do having student peer review English papers to encourage participation is not at all "wrong" or "unfair" to the students.

I would also suggest asking the students why they didn't do it and consider not doing the assignment in the future. You could also bump the essay up to 3X its current length and make it worth 10 points since people don't want to do it. I'm sure the students will not want the alternative and have no problem doing the assignment from then on. Then, for those who did the assignment like you asked you could somehow figure out a way to not come down so hard on them by giving them a score boost or something or by flat out saying "the one's who did the assignment only have to do a regular sized essay".

There are plenty of things you could do to fix this. In fact, you could even just give extra credit assignments that are reasonably difficult. Then, those who just felt uncomfortable reviewing will end up still getting a good grade because you will see that they are willing to do the work and just didn't feel comfortable with the assignment. Of course, there is only likely to be 1 or 2 people like that in the class. Most of the people will probably just blow off the extra credit and end up with a not-so-good grade in their homework.

Finally: don't worry about it too much

There will be students who are just lazy or don't want to do assignments. At some point, it will become clear that they just won't change no matter what you do. At that point, they've made their bed and they'll have to lie in it. I'm sure you gave your students a grading metric of some kind at the beginning of the class. If they end up with a bad grade or fail, that is ultimately their fault.

  • I don't think you read that right. Note that the OP says "By default, they are giving their peers F's on their assignments" This is the fundamental problem: people are getting graded based on what their peers do or don't do. – dmckee Oct 30 '16 at 17:17
  • @dmckee She didn't say the peers actually got an F. She said that the students are rating their fellow students with an F. It doesn't mean the fellow students actually get F's, just that the other students are by default saying they hate their peers work which is a very negative attitude to have. – The Great Duck Oct 30 '16 at 18:26
  • @dmckee I already said my opinion regarding using the F grades as the reviewed students grades. I flat out said that if it were the case then I recommend the students immediately go to their academic advisor and report the teacher for bad conduct. I was giving the asker the benefit of the doubt and assuming that maybe some people misinterpreted it. Because when I read it, I merely thought she was saying the students were just doing peer reviews to demonstrate reviewing skills, and that the students were not doing it which implies they hate fellow students work.Also,not grading = no grade not F – The Great Duck Oct 30 '16 at 18:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.