3

I am currently working as a graduate teaching assistant for a semester-long introductory physics course, aimed primarily at Biology/Pre-Med undergrads. The students usually have a short quiz every two weeks in a discussion section, along with two prelims (mid-terms) and one final exam. The TAs make the grading schemes for their own quizzes as each discussion section has a different quiz.

One thing that I find very strange with the grading (quiz or prelim) is that the grading scheme is made after seeing the answer sheets submitted by the students, not before. The intention behind doing so (at least for the quizzes) is to assign less points to the harder questions and more to the easier ones so that the average is roughly 7.5/10 (this is a soft rule). Similarly, on the prelim, the points for individual sub-questions were decided after conducting the exam and seeing a few answer sheets (points for each question were fixed beforehand).

  1. Is this a standard practice for introductory physics courses in the US?

  2. What are the benefits of this practice over deciding a grading scheme beforehand?

From an answer describing a good grading system:

At minimum, a good grading system should meet three criteria:

  1. it should accurately reflect differences in student performance
  2. it should be clear to students so they can chart their own progress
  3. it should be fair

On the first two counts, making the grading scheme afterwards seems to be a bad choice:

  1. Accurately reflecting differences in student performance → making the grading scheme later may bias the grader towards assigning higher points for conceptually easy questions instead of more challenging ones.
  2. Clear to students → the points for individual sub-questions are not assigned beforehand so students cannot judge how much time they should devote to different problems.
  • 1
    In my experience, tests and quizzes have a number in parentheses indicating how many points the question is worth. But if the TAs are inexperienced, I can imagine that they might be feeling their way as they go. – aparente001 Mar 14 '17 at 20:24
  • I thought so too but students usually end up with much lesser scores with that method (~33%), although it could be because I am a relatively harsh grader. – cutculus Mar 14 '17 at 21:11
  • Maybe it would be helpful to give them a practice test to do at home, and to go over together in class. – aparente001 Mar 14 '17 at 23:49
  • 1
    @DanielR.Collins, do you have suggestions for improvement given the constraints (like roughly maintaining the average)? I find that if I make a grading scheme beforehand, students end up with bad scores like 4 or 5 out of 10 (so I've never actually used it). – cutculus Apr 21 '17 at 2:34
  • 1
    This is my first semester for this course as a TA so I'm still learning. Most often I end up underestimating the difficulty (or are my students underperforming?). I will try your suggestion. Thanks! – cutculus Apr 21 '17 at 2:40
2

This seems perfectly reasonable to me. In this type of freshman physics course, there are some very common difficulties that students have:

  • Physics is counterintuitive, e.g., students often intuitively believe in some kind of half-baked aristotelianism, and this belief can be resistant to instruction.

  • Students try to solve problems by shopping for a formula that has the right letters in it, and then plugging numbers into the formula. They don't think about what the formula means, when it's true and when it's false, etc.

This is all very well documented in the pedagogical literature. The best summary I know of, although it's out of date, is in Mazur's book Peer Instruction. Mazur shows a lot of evidence that physics instructors are in denial about what their students are really learning and about the (in)effectiveness of traditional teaching methods. There is a huge gap between how physicists think about their subject and how their students think about it.

A lot of these issues are qualitatively different than in other fields, such as mathematics. E.g., I don't think students have deeply held intuitive preconceptions about derivatives and integrals before they walk into a freshman calc course. In a course like freshman calc, a lot of what the students are learning to do really is just algorithmic -- there's a reason it's called calculus.

For these reasons, it can be extremely difficult for an instructor in a freshman physics course to write appropriate exam questions. Results can be really horrible if you write questions that require conceptual understanding, multistep problem-solving, symbolic rather than numerical calculation, or interpretation of symbolic results. You can write problems that just require plug-and-chug, but then you're dumbing down your educational standards.

Because of the gap between how instructors think about physics and how most students think about physics, there is a real danger when setting an exam that you will be taken by surprise by how horrible the results are. This is particularly likely to happen with inexperienced instructors. Fiddling with standards after taking a look at the answers is IMO not an unreasonable way to prevent this kind of meltdown. Especially for a non-tenured instructor who can't afford bad teaching evaluations, I think it's pretty understandable to do this. As time goes on, people usually get a better feel for what kinds of questions will be hard. If I were evaluating someone for tenure, I would rather see them doing the kinds of things described in the question than see them writing plug-and-chug exams.

There is one thing that does seem a little odd to me about the specific situation described in the question. If I'm understanding the question correctly, the instructor in charge of the course seems to have delegated the writing of some of the lower-stakes quizzes to TAs, but the instructor will still be the one writing the final. However, there doesn't seem to be any mechanism for making sure that the standards and style are consistent throughout the course. Ideally students would be doing homework that is similar in style and difficulty to the quizzes, prelims, and final. The instructor should be setting the tone as to what should be the mix of easy and hard, concepts and mathematics, numerical and symbolic. If that isn't happening, then something is wrong.

Daniel R. Collins wrote in a comment:

In future semesters adjust the tests so that's not needed. Personally, I don't find it hard to dial in an appropriate difficulty level after the first semester.

If these questions are being written entirely by TAs, with almost no supervision by the instructor, then it seems unlikely that there is any institutional memory that would allow this type of adjustment to occur over time. If that is the case, then the serious problem is lack of supervision, guidance, and institutional memory -- not a particular practice in grading quizzes.

0

This seems like a variation of norm referenced grading aka grading to the curve. What is unique in your question is that the TAs are not using the student's responses to set the curve for the overall score of the quiz but rather to set a curve for each individual question. This is strange in my personal experience but I cannot say definitively that it is an unacceptable practice. However, probably no one with thorough training in assessment in education would do this.

One of the concerns you mentioned was fairness in the grading. One way to address this is to clearly communicate how quizzes and exams are assessed and graded. If you explain the current system it will help significantly. If students know what to expect they may disagree but it reduces the risk of complaints of unfairness

  • How can you not use the students' responses to set a curve for each individual question? Out of basic fairness, you want to make sure that students who make the same mistake get the same grade on each question. And unless you've given the same question many times before, you won't know what the common mistakes are until you see them. – Peter Shor May 15 '17 at 21:41
  • @PeterShor: Isn't that the whole point of a "grading rubric" -- to set expectations, to show examples of good and bad work, and to practice on those types of questions in advance? Or do you think that grading rubrics are infeasible in principle (possibly for your specific discipline)? en.wikipedia.org/wiki/Rubric_%28academic%29 – Daniel R. Collins May 15 '17 at 22:43
  • 1
    @Daniel: Grading rubrics prepared in advance are a good idea, but they have their limitations. What happens when 20% of students make exactly the same mistake, and it's not completely clear how to score it from the grading rubric. Does this not happen in your discipline? Or do you just let the graders judge how to grade it the best they can from the grading rubric, and not worry about consistency? – Peter Shor May 15 '17 at 23:11
  • @PeterShor: Thanks for the response. I'm at a large community college teaching things like basic programming, statistics, and algebra. I don't have graders other than myself. In any problem I just make an answer sheet and assess points proportionally for correct work done, usually 2-6 points per problem, and I don't find any mistakes that fail to fall into that rubric. Usually I can predict in advance what the average grades will be in about a 10% margin or less. That prediction does not always include the 75% point, nor do I feel that it needs to. – Daniel R. Collins May 16 '17 at 1:14
  • 1
    @Daniel: Yes, if you're the only one grading the papers, maintaining consistency is a lot easier. – Peter Shor May 17 '17 at 11:42
0

There are definite advantages to this scheme, but I see one big potential pitfall.

The advantages: sometimes a question you pose has a "garden path" wrong answer where students can follow what seems like a perfectly good line of reasoning, and end up not getting it correct. Maybe attacking it from this approach would work in principle, but you end up with horrible equations you need to solve. Whether a student gets these questions right might be rather random, depending on which approach they happen to think of first. If you reduce the number of points these problems count for, the grading will be fairer.

The potential pitfall: it will be very tempting to look at what the two or three best students in the section did, and use this to calibrate the questions. This gives an enormous advantage to the students you think are the two or three best students in each section, and you should try to avoid doing this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.