38

It is common practice to have a homework strategy as follows: a professor might assign N homework problems but only have some random sub-set of those problems graded. This is motivated by a large class size, and a finite amount of grading time. The argument is that it motivates the students to do all the whole homework set, as they will not know which ones will be graded.

However, as someone who has been assigned to grade in this way, it feels immoral. I have had students who will do 9 out of 10 of the problems, but that one unanswered problem is 1 of the 3 I am grading, so they get a low grade.

I am curious to hear what folks here have to say about such a practice.

1
  • Comments are not for extended discussion; this conversation has been moved to chat. Please read this FAQ before posting another comment.
    – Wrzlprmft
    Apr 15 at 8:15

18 Answers 18

83

While it may average out to be alright, it is very important to look at the other side and consider how a student perceives this grading practice.

Students don't tend to look at the average but on the outliers which are caused by this system of grading things. The attention is shifted to those that lucked out when giving a partially solved assignment and those that hand in a very well solved assignment and barely/poorly pass. Particular attention is shifted towards those that experience one extreme or the other multiple times.

From a student’s perspective, luck becomes a factor in something simple as having their assigned work to be looked at and this is simply frustrating regardless of whether or not it evens out or can be considered fair across a semesters worth of assignments turned in.

1
  • Comments are not for extended discussion; this conversation has been moved to chat. Please read this FAQ before posting any further comments.
    – Wrzlprmft
    Apr 15 at 8:07
25

I see no ethical issue, particularly if the students are aware of this grading scheme in advance. The defect you observed averages out: the unlucky students this week might take no penalty for partially-completed homework during another week.

Still, if you are allowed to modify the grading scheme, you could consider mitigating the effect (or perceived effect) of luck by giving “completion points” for plausible-looking answers to the other questions.

1
  • Comments are not for extended discussion; this conversation has been moved to chat. Please read this FAQ before posting another comment.
    – Wrzlprmft
    Apr 15 at 8:25
22

While I see no ethical issue here, you could suggest the following alternate scheme.

Once the homework is due, you take a quiz where students are allowed to use their homework notes. In the quiz you ask the students to provide solutions to the (randomly) chosen 3 problems in a limited time.

As a result, the graders only need to grade the 3 chosen problems.

If the students have done the problems before, they can write down the solutions from their own notes. If they can solve the randomly chosen 3 problems without having done any of them before, in the limited time available, then they probably deserve to get a good grade!

2
  • 3
    I've seen this done and I think it works well and addresses many of the involved issues - for the student, the homework becomes more optional and worth discussing (which is where the real student gains often are), while at the same time, the TA/professor grading and academic honesty loads are greatly reduced! (note that I recommend having problems of the same style, not precisely the same to encourage understanding rather than wrote copying)
    – ti7
    Apr 14 at 16:01
  • See also my Answer here academia.stackexchange.com/a/116463/68971 with a comparable strategy (homework discussion and any-paper tests)
    – ti7
    Apr 14 at 16:05
10

It is not only ethical, it is an excellent way of assigning homeworks especially if homeworks are necessarily long and the number of students is large.Say, if a homework assignment has 20 problems (normal number for a calculus course) and the class size is 30 students, then you would have to grade 600 solutions.

Another thing is that the homeworks for a class of calculus level consist mostly of practice problems, many similar problems designed to practice certain skill. Usually it is clear whether a student has acquired this skill if you grade one or two of these problems.

6
  • 2
    20 Problems is normal for a calculus course? I think we had maybe 5-10 proofs per Homework assignment per week
    – SirHawrk
    Apr 14 at 8:00
  • 1
    @SirHawrk: 5 proofs and 15 computational problems (find an integral/derivative/limit/sum of a series...).
    – markvs
    Apr 14 at 8:21
  • 3
    I don't think we ever had Computational problems that were not just part of proofs
    – SirHawrk
    Apr 14 at 8:34
  • 4
    @SirHawrk: What you are saying about your "calculus" course is highly unlikely.
    – markvs
    Apr 14 at 8:42
  • 6
    @markvs I don't think SirHawrk is lying—it sounds very similar to intro calculus at Caltech, for example. Apr 14 at 18:36
8

With the scheme of picking 3 problems to grade, if have an equal probability to grade any problem, then a student who regularly does 9/10 problems should expect you to grade the one they didn't do about 30% of the time. That is something a student should expect to happen fairly often, if they turn in, say, 10 homeworks over the course of a semester. Even for one homework, that is a fairly large risk, that they are taking by not doing the question.

There may be correlations that make this more likely, for example if students tend not to do the more difficult questions and there's a bias toward grading the more difficult ones.

Given that the policy is announced in advance and grading the one unfinished problem is not expected to be a particularly rare outcome, the policy seems fair to me.

Based on some discussion in the comments, let me add I don't think this policy is ideal or good. It clearly has problems in that there is some noise in the evaluation of the student. But, I can easily imagine situations where some instructors would choose this grading style as the least bad option. In particular, I think this scheme can make sense in a scenario where you have a lot of students, limited resources to grade, and a lot of (mostly straightforward) homework problems and assignments. Then, the noise should be fairly small (at least comparable to other sources of noise inherent in grading such as the variation in harshness/generosity of different graders and the choice of a finite number of problems to put on the homework), and it could be used by the instructor to encourage the students to do all the problems necessary to cover the full range of material, while not overspending resources on grading.

14
  • If this is a math course, figuring out this risk could be considered part of the class. It seems like appropriate for an English class.
    – Barmar
    Apr 14 at 14:02
  • @Barmar (assuming you meant to say "it seems inappropriate for an English class): It seems strange to think of assigning 10 problem homework in an English course instead of a paper. But, anyway, in my mind the core of the issue is that if the student simply does all the assigned homework, none of these issues ever arise. They only arise because the student is taking a risk by not finishing an assigned task. If so much homework is assigned that it's impossible to finish all the problems in the time students are supposed to commit to the course, that is an issue, but that's a different topic.
    – Andrew
    Apr 14 at 15:27
  • 5
    But what if they do all 10 problems and just get 1/10 wrong? Then they're a 90% problem-solver, who could still get unlucky that the few they answered incorrectly will be in the set of graded problems. A student who did 100% of the work and solved 90% of all the problems shouldn't get a 70% grade. This does not seem a fair system. Students also have limited time and energy, not just instructors. If a student only had to do 3 problems instead of 10, more time could be spent on each one ensuring it was correct. The student should be able to economize just like the instructor does.
    – ErikE
    Apr 15 at 0:56
  • 1
    @ErikE Every system is unfair. If only three homework problems are assigned, perhaps there will not be enough problems to cover the full range of material, and then the final will have a problem that wasn't done in one of the homeworks and people will complain about that. Or, the three problems chosen may cover concepts the students doesn't understand, while they could do a problem not chosen. (...)
    – Andrew
    Apr 15 at 1:07
  • 1
    (...) Really, for a course with a lot of homeworks where the homeworks are 20% or less of the final grade, the variance introduced by this system is not big. If the student's goal is to learn the material, they should do the work. For some courses (particularly introductory courses), giving three "big" problems is less productive, because the point of the course is to learn relatively straightforward exercises that are building blocks to higher level material. The way to learn that is practice.
    – Andrew
    Apr 15 at 1:08
6

The grading scheme may be fair under certain conditions.

  • You need many homework assignments. If it is only biweekly or monthly, then averaging out is unlikely to occur for almost all students.
  • The students should always finish approximately the same number of problems on each assignment. Imagine that you have 10 assignments with 10 problems each, for a total of 100 assignments. You grade one random problem of each assignment. Now, one student one student solved 9/10 problems on assignment 1, and all problems on all further assignments. The student thus solved 99 out of 100 problems. Unfortunately the missing one was the problem chosen to be graded on assignment 1, so the student has 9 correct assignments and one missed one, for a total of 9 out of 10. Sounds unfair to me.

Furthermore, you might run into trouble with perceived unfairness.

The larger the number of students, the more likely it is that one will hit a lucky/unlucky streak. This will quickly spread amongst the students and they will perceive the grading as unfair, even though it is fair for the vast majority.

And finally, you might be accused of not really choosing the questions randomly, but selecting only problems which your favorite student has solved. This could be prevented by publishing the selection to be graded beforehand in an encrypted manner and then handing out the encryption key/password after the grading has occurred.

Checking whether this grading system is fair/unfair under different conditions might be a fun exercise to some introductory statistics class.

Personally I would recommend to tell the students beforehand which exercises will be graded and which won't. After all they chose to be there to learn I assume. For me it worked pretty well with pretty much no graded exercises at all. Most of my friends and me did almost all exercises anyways. And the ones who didn't: Well, most of them stayed at university for longer, only to not graduate in the end.

Edit Forgot to mention that we got feedback on all exercises, graded or not. Sometimes individually, sometimes in the form of example solutions. I think that's important so that all solved exercises help the learning process, not just the graded ones where you get feedback in the form of the grade.

3
  • 1
    Personally I would recommend to tell the students beforehand which exercises will be graded and which won't. --- You could also tell students that some (many, all, etc.) exercises on their major tests/exams will be selected from the exercises that were not graded. Of course, this works better if there is a reasonable uniformity in the types of exercises, and the relevant types are represented in both graded exercises and ungraded exercises. Apr 14 at 12:47
  • 3
    "And the ones who didn't: Well, most of them stayed at university for longer, only to not graduate in the end." This sounds like a very strong argument against following the recommendation of telling the students which exercises will be graded and which won't.
    – Stef
    Apr 14 at 13:40
  • @Stef I think that depends on your goal and your target group. If you have a bunch of high school students and want to make sure everyone learns some math, then sure, I totally agree with you. If you have a bunch of college students who you want to raise to decent adults who take responsibility for their choices and actions while also teaching math, then I think my proposed approach has some benefits.
    – laolux
    Apr 15 at 1:21
6

It is statistically absolutely unethical in the parameters you described, if you pass/fail students based on average assignment grade. I assume a hypothetical best-case example where you:

  1. Teach a motivated class filled with students that do not skip any questions. (The problem only becomes worse if they skip some questions.)

  2. You teach a course with weekly assignments, maximizing the chance of "averaging out". I will assume a 12-week course, so 12 assignments.

  3. Grade objectively and fairly, where each answer is either correct or wrong to give a % score on each assignment.

I assume that an average homework grade of 60% is needed to pass.

Let's model an adequate student as someone who answers ~70% of the total number of questions correctly, in three scenarios:

  1. 12 assignments where 3 out of 10 answered questions get graded (as you mentioned in your post).
  2. 12 assignments where 10 out of 10 answered questions get graded (extra workload).
  3. 4 assignments where 9 out of 9 answered questions get graded (same workload as #1).

The only randomness I allowed in the model is that the correct/wrong answers are shuffled between tests (or within a test for grading subsampling). That is, there is no randomness in the total number of correct answers given by a student in this model. In scenarios 1 and 3 the total number of correct answers given is 25/36, in scenario 2 it is 84/120.

In both scenario 2 and 3 across 100,000 course simulations the student who answered ~70% of the questions correctly has a 100% chance to pass (that is because the average of averages still has a linear relationship with the total number of correct answers, if not subsampling). In scenario 1 this drops to ~94.4%. If all your students are perfectly adequate, you would unfairly fail 2 students out of a class of 37.

What about a more borderline student, one which answers ~61% (22/36, 73/120) of all questions correctly? What is their probability of passing? Without subsampling it is once again 100%. With the subsampling in scenario one it drops to 56.5%!

My code for the above simulation.


Subsampling is fair if and only if you subsample randomly (without bias) and you compute the average grade only over the subsampled questions. It is unethical in my opinion if you compute the average grade over assignments where it is random which questions are graded, as it unnecessarily introduces statistically significant variance in who pass/fail simply based on whether they got graded on the questions they got right, and which they got wrong.

To spell this out crystal clearly: when you use the average assignment grade rather than average correctly answered questions in combination with subsampling, you are throwing away information you already have and unnecessarily replace it with randomness. It's no more ethical than ignoring the existing pass/fail policy and instead flipping a coin to pass/fail students with a borderline grade, as that is essentially what you're doing.

1
  • Nice! But I would add a standard error and use the lower bound of a confidence interval to set the cutoff for getting the credit. Also, consider a process in which those with marginal or low grades can appeal for another random problem to be selected.
    – Elin
    Apr 23 at 21:12
3

An ethical issue would arise if such randomness were the primary cause of poor grades for a student who had no knowledge of the rules and no way to make up a poor individual grade. It is the overall course grade that matters, not the grade on a single assignment.

But, since this is homework, not the final exam, the student can probably absorb a single hit and still come roaring back. Grades are usually based on a collection of individual marks, even when only a final is graded.

But, a student who is otherwise qualified shouldn't be given a poor overall grade due to any random factors. A fair assessment, overall, needs to be made.

4
  • 3
    "cause of poor grades for a student who had no knowledge of the rules" - This is either entirely the student's fault, or there has been a disastrous error in the beginning of the course, namely not telling the rules to the students. Which is obviously wrong.
    – Neinstein
    Apr 14 at 8:57
  • @Neinstein, indeed it is "obviously wrong" to not tell students what the grading criteria is. So, I was stating a general principle. I don't know the actual specifics here. And, I hesitate to assign "fault" to busy students who have many tasks to manage and, perhaps, limited time for it. There is a problem with any form of "gotcha" grading.
    – Buffy
    Apr 14 at 12:57
  • 4
    It doesn't seem to me there is actually a large random component, because there's a simple way to guarantee that an individual homework grade won't be affected by this policy: just do all of the assigned problems. It might be that the teacher is assigning so many problems that it's not reasonable to complete all 10 in the allotted time, which would be unfair, but in my mind that's a separate issue.
    – Andrew
    Apr 14 at 14:42
  • @Buffy I still fail to see how the student missing core information regarding the course he is attending to, due to being busy with university stuff, is an ethical issue with the instructor. I just don't see how either of the two causes outlined in the first paragraph would support that the grading system is ethically wrong.
    – Neinstein
    Apr 19 at 14:22
2

Another way to do this, which partly mitigates the 'luck' problem, is as follows: a student can hand in A of the N exercises. Three of them will be graded randomly. If the score on them is x%, the total score on the assignment will be x%*A/N.

This way, a student that solved 7 of the 10 exercises cannot get a low score by having bad luck: if the 7 exercises were indeed correct, the score will simply be 70%. Conversely, a student that solved 3 of the 10 exercises can not get more than 30%. Luck only comes in is if a student tries to game the system by intentionally handing in wrong answers, or if a student thinks they solved an exercise, but they did not.

The disadvantage of this system, is that, while you only have to grade 3 exercises per student, it will not be the same 3 each time. So there is some extra work in devising grading schemes for all exercises etc.

4
  • 2
    "Luck only comes in is if a student tries to game the system by intentionally handing in wrong answers" If I were a student and a teacher set up this weird system, I wouldn't even think of that strategy as "gaming the system". I would think of it as "This is obviously was the teacher wants us to do so of course I'm doing it". Although I would not exactly intentionally hand in wrong answers; I would intentionally hand in random answers or incomplete answers.
    – Stef
    Apr 14 at 13:43
  • That wouldn't give you a higher expectation value of your score, though.
    – Squala
    Apr 14 at 16:15
  • Interesting! And counterintuitive if that's true. I'm about to turn my computer off but I'll think more about it later.
    – Stef
    Apr 14 at 16:35
  • 1
    @Squala Only if the wrong answers are all 100 % certainly worth zero marks. Extremely poor answers that might be worth a mark or two still increase EV. Apr 15 at 8:00
2

The students are told in advance, so they know what gamble they are taking leaving one or more unanswered. (If they don't that is an academic skill in itself that needs some work!)

So in principle, this is a way to ensure that all students take the material covered by the set equally seriously.

So the practice as such is ethical. There is a problem, however, in that some academics wish for their subject to occupy more of the students' attention that is proportionally reasonable. (The "my subject is the only one worth doing" illness is too prevalent among academics.) They can achieve this via this trick. If I have three contact hours per week (normal for a typical module) and the students have 15 contact hours (excluding TA time, workshops, tutorials etc) then my homework should roughly occupy 1/5 of their homework time. If I inflate this doing the subset trick, then that is unethical.

Also, the prof should decide (perhaps using randomising methods) which questions are going to be marked before seeing the worked answers, and not taking any bribes from students.

1
  • I agree and think this is an important point. Giving too much homework so that it's not possible to complete the assignment without spending more than the hours that a student should reasonably commit to the course, is a problem. But I think that's an independent issue (except insofar as people may use the "grade a subset" policy to enable giving too much homework, as you said).
    – Andrew
    Apr 14 at 17:36
2

Consider that essentially the same thing happens each time a student writes an exam in that only a subset of all the possible exam questions are actually graded, while the remainder don't appear on the exam. The difference is simply the amount of information that everyone has: in the exam scenario the ungraded questions are unknown, while in the homework scenario the ungraded questions are known.

If time constraints are such that only three questions can be graded, then the other option would be to just ask three questions. In this case one could worry about the student who knew 70% of the material, but got 0% because the question selection didn't reflect their knowledge. Such a student would have (likely) done better in the randomized version. Finite time requires that we approximate student knowledge via the random selection of questions from an infinite pool. As such, we must accept that the randomization will be beneficial for some and a detriment to others (and hope that this evens out over the course of a semester).

To me, asking 10 and grading three is ethically equivalent to simply asking three.

5
  • 1
    "To me, asking 10 and grading three is ethically equivalent to simply asking three" how? If I ask you to grade 10 exams and then tell you that only 3 actually needed to be graded, would that equal to simply asking you to grade 3? In the homework scenario, assigning a few hard problems that all would be graded would incentivize students to study and figure the problems, while assigning a bunch of problems and then grading a few disregards that they also have limited time (just like grading time.)
    – jDAQ
    Apr 15 at 18:59
  • @jDAQ Your concern seems to be that asking 10 and grading 3 wastes student time so is unethical. Assuming that the professor isn't asking pointless questions just for filler, then I disagree. If all of the problems have a purpose towards satisfying learning outcomes and preparation for an exam, then the effort isn't wasted and will contribute to the course grade, but in a delayed manner. If it's only ethical to expect students to do work that immediately becomes a part of their course grade, then we wouldn't expect them to study for example.
    – Scott H.
    Apr 15 at 22:01
  • @jDAQ Do you see no scenario when those "extra" seven questions would be beneficial for the students to do? What if (variations of) all ten were to appear on the final exam for example?
    – Scott H.
    Apr 15 at 22:05
  • 1
    I see scenarios where the grader could at least give completion points to problems they assigned, were solved, and they do not have time to grade. Instead of relying that "on average" people got a grade (highly) correlated to the quality of their work. Or, the graders could let them know half the problems that will be graded and make students' grades less probabilistic. As an incentive to make students do their work this is effective, but not as much as feedback on the quality of their work (grading 3/10 problems). They could grade just 1, it would be equivalent to simply asking 1, no?
    – jDAQ
    Apr 15 at 22:43
  • @jDAQ I think your suggestions are good, and improvements on the scheme can be made. I just don't think that it's (more?) unethical/immoral, per se, to grade m of n assigned problems with m<n rather than grading all of them.
    – Scott H.
    Apr 16 at 0:40
2

Let's just count. We have 10 problems out of which 3 random ones are graded, so the probability of any particular homework to be graded is 0.3 and the direct computation shows that the largest possible variance on a set of 10 problems for the grading score of an individual student is 7/12, attained if the student solved exactly 5 problems, which (50% performance) is the usual cutoff for the F grade. The typical semester course lasts 16 weeks with 2 homework sets per week, which gives 32 homeworks. Assuming the normal approximation, we conclude that one standard deviation is at most $\sqrt{\frac{7}{12}32}=\sqrt{56/3}<4.5$ problems even for the borderline students (everybody else is better off). Now, with the expected score of 48 on the borderline, the probability that a (barely) passing student will get <44 problems is 16%, which may be a bit too much. If you lower the threshold to 39 problems, being out of luck has probability 2.3% for each student, which seems acceptable for moderate size classes. If you have 100+ students, I would go for 3\sigma, i.e., the passing score of 35 problems with probability of being unlucky 0.1% for an individual student. In other words, if you add 14 points to everybody's score in this scenario, the bad luck (getting a lower grade than one deserves due to the random chance) will be essentially wiped out even in a large class. You, of course, may now raise some scores beyond what is deserved, creating another type of "moral problem".

In other words the message is that the idea of such grading is neither ethical, nor unethical by itself. It just has a certain chance of error that you should clearly understand and, if you find it unacceptably high, compensate for before implementing this scheme.

2

I once went to a pedagogy talk and the faculty presenter (in a STEM discipline) said that, among things, which can improve student results are: assigning homework, collecting homework, grading homework.

You have to think about "what is the purpose of this specific homework?" and "what is the purpose of the grade?" Is it to ensure understanding, correct misunderstanding, deepen understanding, assess understanding, and then what is the level of the work (think Bloom's taxonomy). There are many kinds of homework, and different strategies are okay for different types. For example, in teaching writing a "minimal grading" model for grading, e.g. essays, is often suggested for both pedagogical and time management reasons. It's good for students to write, and it actually is not necessarily helpful for them to get back a document filled with red ink. But if you were giving a multiple choice grammar test obviously it is minimal grading by definition.

One purpose of grading is "riding herd" which is to say to simple establish that the student has done the reading or made a valid attempt at the assignment. Then it's fine to just quickly give check/no check grades.

In your case, it sounds like the grading involves giving substantive feedback in order to help students do better in the future and highlight errors in their thinking. It sounds like you would be giving "partial credit." Thus all of the comments about "getting it right" are not exactly on point because it is not an all or nothing grade.

In that case, it makes sense to randomly pick a few problems to do that for. That work is time consuming and students are unlikely to read all of the feedback on all problems.

Of course if the homework is a high percentage of the grades and are the main summative assessments of student understanding that's a different story.

So, if the main benefit of homework is improved understanding then having students do homework is good. If they need a reward for doing homework, announcing that three problems will be randomly selected for grading on each assignment lets them know they need to at least try all the problems. You could also reward simply attempting all of the problems on a check/no check basis.

1

Ethical? Not fully sure about it, I do not see a problem with this. Fair or sensible? For sure not.

The function of grading is grading. The function of homework is training.

Bias: This scheme results in better grades for students who do all the homework, even if they would perform worse when presented with identical tasks as better students who - for some reason - don't have the time to do all the homework.

Adding spread: If the grade should be an assessment for how good the student is able to handle problems, then this is going to increase the statistical spread of the assessment.

If you have to use uncertainty about what is being graded as a whip to make students work, something is wrong with your lecture.

2
  • Hmm why would something be wrong with your lecture? Do you think it's possible to enthrall every student with your presentation so much they will go and learn on their own?
    – DRF
    Apr 15 at 17:33
  • @DRF: if you want that people do the homework, make handing in homework and prerequisite for passing. If you accept homework, then correct it all. If you correct it all, then you can grade on every homework if you li. If you give a grade on every homework, then it would be irrational to not use all the information available to grade the students. If you use uncertainty in getting graded as an means to do your work half-a**ed, you are not making the best use of the students time to learn.
    – Sascha
    Apr 16 at 13:08
1

It is ethical so long as problem sets are not a substantial part of the final course grade. The pedagogical value can be increased, and concerns about unfairness mitigated, through some minor adjustments.

I use a version of this in some of my courses. The major concern that you raise is one of unfairness due to the randomness of which problems are evaluated.

The ethics of the assignment should be evaluated with an understanding of the difference between formative and summative assessment. Formative assessments are supposed to provide feedback to the student and the instructor about the student's (and the class's) progress in the material. Summative assessments are supposed to provide an evaluation of student mastery at the end of a course module or entire course.

Summative assessments are high stakes (i.e., have a substantial impact on a course grade) and can include things like term papers and examinations. Randomness itself in a graded assignment is not a concern, but would become a concern for a high stakes assessment. An example where I think it is handled well is that, as an undergrad, I often encountered essay exams (usually in humanities or social science courses) where we were given 5 questions to prepare for, but only 3 appeared on a final exam (and sometimes the student would only have to answer a subset, like 2 out of 3). This offers a nice balance between having an overly long exam vs. a short exam where the students study to the test, i.e. ignore material that will not be on the exam. An extreme in which it is handled poorly would be assigning three papers and grading only one of them, where the final paper represented 30-50% of a final course grade. This would be indefensibly arbitrary, and I've never seen anything like it done.

Formative assessments should be low stakes, and some instructors seem to think should be no stakes (i.e., not graded at all). The question is how do we treat problem sets, which are common in STEM courses where practice of methods is necessary to learning.

Problem sets sit in an awkward place. They are often graded, and sometimes form a substantial part of a course grade, yet their place in a course sequence really means they should be treated as a formative assessment:

Reading/Lecture → Problem Sets → Examinations

In some sense, if the student does well on the summative assessments, the formative assessments shouldn't drag their grade down. That is, if a strong student is lackadaisical about homework but aces the exams, giving the student a poor grade due to missing or sloppy homeworks seems both punitive and just plain inaccurate as an assessment of their capabilities. So if their purpose is formative rather than summative, problem sets could be ungraded, and used entirely to provide practice for the student and feedback on how well they are learning the material. Unfortunately, I have found that if the problem sets are ungraded many students won't do them, and subsequently will perform poorly on exams.

Sidebar: Doing the problem sets improves student performance, but inevitably there seems to be a high correlation between the students who don't do the problem sets and those who I suspect would struggle with the material no matter what. I think the causality is that students who are underprepared get discouraged by the problem sets, and then don't do the work they need to in order to master the material! Trying to encourage those students while still making it clear that they have to actually do the work is a constant struggle, and probably not unique to me.

The students have to do the problem sets in order to learn the material, but the grades are really utterly irrelevant to the assignment's pedagogical purpose. But getting the students to engage with the material is very difficult if they are no stakes assignments. Offering low stakes (maybe all the problem sets are no more than 10% of the final grade in total) is probably enough motivation to get them to do the work. Since the purpose is practice rather than summative assessment, assigning more work than will be graded is acceptable. It becomes less acceptable if the impact on the grade is higher. It also becomes less acceptable if nothing is done with the ungraded problems. That is, you shouldn't just grade some of them and completely ignore the ungraded ones. Ungraded problems can be gone over in class, in discussion section, or answers could just be provided for the students to self-check.

Finally, there are a number of ways that the assignment could be changed to increase the pedagogical value.

  1. All problem sets can be graded for extra credit. I do this in some of my courses. I have found that it provides sufficient motivation for most of the students to attempt the problem sets, while mitigating student concerns regarding grading of random answers. (If they don't earn the points, it's "just" extra credit, and has no impact if they do well on other assignments.)
  2. For your example, points can be awarded for 3 out of 4 possible answers. That way, if a student skips (or just messes up) one question out of the 4, that one will be ignored. Points will be awarded only for best 3.
  3. Switch to the flipped classroom model. This is of course a major pedagogical shift. In this case, the problem sets would be done in class. They could be completely ungraded, as the professor and teaching assistants will be observing the student learning directly (both that they are doing the work, and how well they are doing). Or they could be graded, but students will have assistance with them. Or they could be marked "complete" if the student is present and works diligently, rather than for correctness, but the professor could require them to be submitted for a grade by students who are absent or clearly not doing the work while in class.

Any of these things would improve the pedagogical value of the problem sets. But the randomness in and of itself is not an ethical concern so long as the problem sets are not a substantial part of the final course grade.

1

In lieu of writing more stuff in comments I decided to write up an answer that adresses both the original question and some of the discussion that has emerged following it. I'm writing from a perspective of math and in particular the lower level math courses 300 level ones and below in general (i.e. Calc 1-3 and below in the US). This is definitely not applicable to the European system where this wouldn't be an issue probably.

I will first describe the system I'm talking about in more detail. When I taught we would usually assign roughly 20 problems a week for ~10 weeks of the semester and grade 5 problems of each set. The total grade for a homework set was made up of 5 points for the five problems 2 for completeness and 2 for tidiness. Completeness is self explanatory, and tidiness involved not handing in the homework chewed up and illegible and using correct mathematical terminology (equals signs where there supposed to be equal signs, implication where there were supposed to be implications and similar). The total grade for homework was usually about 20% of the course grade. This meant any given problem was worth about 20/90 % of the total grade. (Much less actually because sadly almost all courses I graded were graded on a curve, don't get me started on that).

I want to answer two things. First, is this immoral (assign N choose K to grade)? Second, is this the best?

Let's start with number one: No, it's not immoral.

The reason we assign and grade homework is mostly because students even at the university level (in the US) can't be trusted to do what they should on their own. If the homework is not graded a significant portion of the students will not do it.So they will not learn and they won't figure out where they have shortcomings. Now the morally best choice would be to treat students as adults assume they do the work and when they find out they are struggling come to office hours or otherwise deal with it. Unfortunately that doesn't seem to work. Thus, trying to provide a service, the system attempts to incentivize doing homework by grading it.

Grading all of the homework is logistically and functionally impossible. It would either mean not assigning enough for it to be worth it, or paying way too much money for grading all of it. Math TA's are usually costly and in short supply since they often have better ways to earn money and the STEM fields are less full. Assigning more homework and announcing which problems will be graded in advance doesn't help since it goes back to many (most) students only doing the graded parts.

I admit you only ensure there's no luck involved if the student does all of the homework correctly. To me that's enough, you have a path to a perfect score that's not based on luck. Also assuming you do all the problems and most of them correctly the chances of getting a significantly bad grade are rather minimal.

In the end you have an optimization problem, to which the answer of randomized grading is a heuristically optimal solution. Thus it's not immoral.

Is it the best? No. I don't believe it is the best.

I do believe given the constraints of the US systems it is probably optimal though. We've already covered why it's not optimal to leave students to their own devices and why not all reasonable homework can be graded. There is another way to deal with the problem of providing feedback to students during the semester though and that's oral exams combined with quizess. Oral examination is (in my opinion) by FAR the most efficient and effective way of both providing feedback and finding out what the student knows. In 10-20 minutes of speaking to a student I get more of an idea of what he knows and doesn't, than I can gain from three exams and 20 homeworks. Unfortunately oral exams are strongly disfavored in the US. I'm guessing it's mostly due to size of class and the dislike of telling students they don't know stuff.

The trouble with oral exams is they are more prone to claims of unfairness or randomness and take more time away from the professor, since exams in the US are usually graded by the TA's, whereas oral exams would likely have to be done by the professor only.

7
  • "Now the morally best choice would be to treat students as adults" I really agree with that. However, I feel you are missing a point in your answer. If you grade 2 out of 10 questions for every assignment, and a student solves all questions for each assignment but only gets 80% correct, then the student has a non-zero chance of failing the class simply because of the subset of problems chosen for grading.
    – laolux
    Apr 15 at 1:43
  • 1
    @laolux This would be the case if the homework was actually worth a significant part of the grade and at the same time the grading was done by automatons without a brain. Since the grading is done by people and the homework is a fairly small part of the grade (read the rubrik we used it's stated at the beginning) it's a non-issue. The TA sees all of the homework often he will add comments to non-graded problems when he thinks it will help. When grades are handed out whether the student got really unlucky (always messing up just the graded problems) would be seen and handled.
    – DRF
    Apr 15 at 6:40
  • fair point, failing is too strong language, but it would still leave a bad taste for the unlucky student. Now I want to stress that it is entirely possible for grading humans to not even be aware that said student handed in solutions to the other problem. At my university for real exams we solved all exercises on different sheets and then put them into different baskets. If the same was done for the homework, baskets 1,2 would be checked and 3-10 discarded (randomly chosen baskets of course), so no one would notice who handed in solutions to those problems.
    – laolux
    Apr 15 at 7:10
  • 1
    @laolux if that was the way the homework was handled, I'd be willing to give you that it's problematic. But homework in all the classes I've seen was handed all together to one TA who graded it. That meant he saw all of it, and would comment on a number of problems. He would only grade the chosen N though. We did also grade on completeness and neatness as I point out above.
    – DRF
    Apr 15 at 7:57
  • 1
    One thing about the "non zero probability" is to (a) review with students sampling and measurement theory to show that while it is possible, with a standard error included it is unlikely over a whole semester that the measure of understanding or effort will be wildly far off. The other is to allow students an "appeal" to substitute another randomly selected problem if they really believe that doing so would impact their grade.
    – Elin
    Apr 23 at 21:06
1

One professor might work this way, and another professor might work another way. Each gives ten homework assignments and grades three. The first one doesn't tell which ones are graded, the second does.

The result is that students either put more effort into your course, which makes it entirely unfair to the other professor. And "putting more effort into your course" isn't a positive. Students will learn a lot from homework they do; they don't learn from the extra work they need to do for optimising the grading. In homework, it is reasonable to focus on the things that the student doesn't know; something that the student knows how to do doesn't help them learning and is therefore wasted effort. Any effort put into your course is effort not put into another course.

On the other hand, the students might vote with their feet and take two other courses instead of yours.

0

If the assignment is to do 10 problems, out of which 3 will be graded at random, then what might perhaps be seen as unfair is that the student might have done a poor job on some of those 3 problems but an excellent job on the other 7. I can see why it would be frustrating to a student to spend a significant amount of time on one problem only to be graded on the basis of another. In particular, if the problem set consists of problems of widely varying difficulty, then this sounds like a sure recipe for inspiring legitimate frustration among students.

On the other hand, if the assignment is to do 10 problems and the student decides to only do 9 and hope for the best, that's entirely on them. They haven't done the assignment and instead intentionally opted to enter a universe where they had a probability of 2/3 of getting away with doing slightly less work than required for the same credit and a probability of 1/3 of being significantly penalized for this. (Significantly in the context of a single problem set. It probably isn't significant in the context of the entire course.) This is a trade-off that they chose of their own volition. Focusing on what happens in 1/3 of all universes where, some could say, they get a worse outcome than they deserve misses the point that there are also 2/3 of all universes where they get a better outcome than they deserve and that the decision to enter into this gamble was their own.

Of course, if this grading scheme is used as an excuse to burden students with assignments which place unreasonable demands on their time, then this would indeed be unethical. But the same is true if all 10 problems are graded.

2
  • 2/3 is actually 0.7, right? (1- the probability that the unsolved problem is graded). This misses one essential point however: we are not talking of students who decide not to do the entire set, but of those who are not capable of doing the entire set, so they don't really "enter this gamble" at their own choice. They do their best and still may get less (or, as you correctly said, more) than they deserve.
    – fedja
    Apr 16 at 1:18
  • 1
    @fedja I agree, but this is a degenerate case of the first alternative considered in the answer, where doing a poor job reduces to not doing the problem at all. Apr 16 at 12:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .