3

I am fresh at teaching computer science. For this subject, students must write an exam. Are there any statistics beyond avg, median, max, min points which you can recommend calculating? Furthermore, can you recommend any visualizations for year over year comparisons or comparisons with various previous years?

Any help is appreciated.

14
  • 2
    Why do you want these statistics?
    – Buffy
    Commented Nov 4, 2021 at 12:19
  • 11
    Trying to do statistics, with no formal education in statistics, is likely to lead to faulty conclusions.
    – GEdgar
    Commented Nov 4, 2021 at 12:53
  • 3
    Your question is perfectly on topic here, but I'd also suggest checking out cseducators.stackexchange.com
    – Jeroen
    Commented Nov 4, 2021 at 13:05
  • 3
    I used to do this heavily (M.A. in stats here) for several years, but never found any real use for it, honestly. There's just too much noise and too many variables in the social system to draw out any actionable inferences. Commented Nov 4, 2021 at 15:48
  • 5
    @GEdgar - to be fair, faulty conclusions only occur in 5 out of 3 cases...
    – Jon Custer
    Commented Nov 4, 2021 at 16:44

2 Answers 2

8

When presenting exam results to students, I usually post a bar chart of the distribution of scores along with min, max, median and mean and call it good. I always look at the bar charts of average scores by question (Gradescope is great for this) but I don't usually post those charts unless there's something interesting I want to share with the class, e.g., that a question turned out to be much more difficult than we expected.

But allow me to soap box for a moment. The exam itself is more important than how you report the statistics.

A good exam has several objectives. Students expect grades so of course it must fairly measure and differentiate your students' performances and give them useful feedback on what they're doing well or not so well across problems of varying difficulty. Plotting scores as a bar chart, you should see a nice bell-shaped curve distribution.

Second, a good exam should measure your performance as well. Assuming different questions or problems test different things, a bar chart of average scores by question can right away tell you which learning objectives were generally achieved and which were not.

Third, a good exam is one last chance to teach or reinforce your learning objectives, e.g., with a problem that asks students to take what they've learned one small step further, resulting perhaps in an ah-hah moment. (One of my staff used to call problems like this "spicy".)

3
  • 7
    I like most of this answer. But I believe that "ah-ha!" moments, while absolutely desirable in general, should not be a goal of timed exams specifically. Students are under enough pressure that they're not going to appreciate any teaching that an exam is trying to do. I prefer to save "ah-ha!" moments for when students are open to them—classes, office hours, or untimed assessment such as homework. (Moreover, leaning into the desire for "spicy" problems often results in exams with negative issues—too long, too difficult, etc.) Commented Nov 4, 2021 at 20:36
  • 2
    I usually post a lower quantile rather than the actual minimum, to spare that student or two.
    – Hasse1987
    Commented Nov 4, 2021 at 21:15
  • 2
    @GregMartin A good timed exam is one where almost all the students run out of answers they could improve before they run out of time. Also, please note the emphasis on "one small step further". It should never be a trick question or one where the ah-hah moment is obscure. A reasonable example would be a recursive problem on an intro CS exam with one more recursive or base case than in the examples from lecture, asking the student to slightly generalize what they've learned. Commented Nov 4, 2021 at 21:30
7

Don't overthink it and don't put too much significance into any set of numbers. There are a lot of reasons for this, but the most fundamental is that any given class is not a random sample from the "population" and the population itself changes from year to year as well as being difficult to define.

Your list of possible statistics is fine. The harder you try to push it the less reliable your numbers will be. And don't put a lot of effort into why some statistics (min, max) vary widely from class to class. Larger samples tend to even out the mean, of course, as expected, but maybe not the mode, or even the shape of the distribution.

But an even bigger issue is that, for a professor, you need to treat each student as an individual, not as a data point. Every student is different, and every student is likely different from you. The very fact that you are where you are, with your educational background and interests sets you aside from nearly all of your students who have different goals than you do. You need to learn to deal with that.

At whatever level you teach you will find that some students find it all very easy and others very difficult. This is aside from differing motivation for what you teach. You need to be willing (and I hope able) to give full marks to everyone in the class. Or to fail everyone in extreme cases. You need to work harder with and for some students than others. You need to learn the tricks to do this efficiently. Statistics won't help you much, and, in particular, they may not say anything about you, though if you need to fail everyone, then you need to reflect on that and make changes.

I've had classes where everyone was (more than) capable of full marks. I never worked harder in my life. I've also had classes where I needed to tell them they might all fail if they didn't change their (learning) behavior. Both sorts of situations can turn out well, though in the latter case I had to take time out to teach them how to learn effectively.

One practice I used in teaching was to look at the grades overall at the end of a course to see whether, informally, the distribution matched what I "thought" the general level of learning was overall. If I thought that the grades "seemed" lower than my judgement of their learning then I would adjust the grades overall a bit upwards, though not downward. This still fulfilled my "contract" with them that if they earned X points their grade would be (at least) Y. Any surprises at the end were happy ones. And yet I was viewed by the students as a very demanding instructor.

But I also realize that when I was a novice instructor I was far too rigid. Experience helped a lot and for many new university teachers there are no pedagogy courses that teach you how to get it right early on.


Edited to add: If your 60-80 students are divided into different sections then be prepared to find that there might be differences between the sections in performance even though the "teaching is the same" for them. One of the hardest things I ever did, and I'm still not sure I was successful, was to teach three sections of the same course of about 30 each "back to back" in three consecutive hours.


There is one statistic, however, though I can't remember its name, that tests the validity of individual questions. It is possible that the statement of some exam questions is confusing or misleading, and the best students to worse than the average on that question. In essence it measures the distribution per question vs the distribution overall. This isn't a measure of student performance, but a measure of the validity/reliability of an exam itself. Perhaps someone with more background in statistics can supply the name of the measure. This one is worth considering.


And if your department constrains what you are allowed to do then see the following: Grades are too high for the department - what should I do?

1
  • 1
    I would like to upvote this answer several times. I hope the OP takes it to heart. Commented Nov 4, 2021 at 15:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .