# What does an optimal grading distribution look like?

Researchers on education must have thought about this, so I hope to find some directions here. Suppose that I am grading students on a scale from 1 to 10. If I have no prior knowledge of the ability of the students but have obtained their test scores (let's say from 1 to 100).

Suppose these test scores have only ordinal meaning. A student with a higher score can be said to have achieved better mastery of the course but having double the score does not mean that one has achieved double as much as another student. We may also assume that a (statistically speaking) large number of students representative of the student population took the exam.

What should be the optimal grading distribution?

We may also assume that the grading distribution/scale should achieve two goals: a) it should be informative about student's grasp of the material, b) it should incentivize students to study the material.

Regarding a) from an information theory perspective, we may want to maximize the information (entropy) of the grade distribution. Thus, we would choose a scale which yields a uniform distribution. However, in practice most teachers implement distributions which are peaked. What is the motivation behind this?

• Hopefully everybody learned everything perfectly, and grades bunch up in the 8-10 range (yes, people make mistakes). Jan 4, 2016 at 2:24
• The optimal grade distribution is the empty distribution --- in the optimal world, people are not motivated by grades, and they do not deceive themselves or others about their knowledge or abilities. Thus no grades are necessary. Jan 4, 2016 at 2:28
• @Boris Even in the optimal world you describe, grades would still be useful. Students aren't necessarily able to measure their own level of mastery, grades are one way for instructors to provide feedback to students on what they've achieved Jan 4, 2016 at 2:35
• The optimal distribution is whatever honestly reflects student mastery of skills for the course. This assumes that there exists a grading rubric which accurately measures relative mastery of desired skills -- which it should, and is a far higher priority than any other concern of the OP's. Mangling the data to fit some preconceived distribution resembles a type of research fraud. Jan 4, 2016 at 5:15
• You may want to look at construct validity. Jan 4, 2016 at 15:22

Assigning grades to fit some "optimal distribution" is misguided. We don't want to maximize the entropy of the grades in a particular course. It's not a very useful measure of a "good" set of grades. To quote from an answer by Anonymous Mathematician:

Strictly speaking, Shannon entropy pays no attention to the distance between scores, just to whether they are exactly equal. I.e., you can have high entropy if every student gets a slightly different score, even if the scores are all very near to each other and thus not useful for distinguishing students.

(Note that this was in answer to a question that asked about using the entropy of exam scores as an indication of how good an exam is at distinguishing between different levels of mastery. It wasn't suggesting that grades should be curved a posteriori to maximize entropy.)

What we actually want is for grades to signal as closely as possible the students' mastery of course material. If every student in the course has achieved truly excellent mastery of the course material, they should all get high scores. This grade then seems to carry very little information. But in reality, it is much more useful to say that all students in this particular class achieved excellence and deserve a 10/10, than it would be to maximize the "information" carried by the grade and give some students a 1/10 because their performance was slightly less excellent than the highest level of excellence achieved by a student that year. This scenario (where all students achieve excellent or very good grades) is not even that unusual, as Michael Covington points out:

In advanced courses, it can be quite proper for all students to get A’s and B’s, because weak students would not take the course in the first place.

For an individual student, the grade should depend on that student's demonstrated mastery of course material, and hopefully not at all (or as little as possible) on the other students in the class.

If you insist on thinking about it from an information theoretic perspective, what we really want is to minimize the Kullback–Leibler divergence between the distribution of students' achievements and distribution of students' grades.

If you have no information about the exam and what it measures, you cannot assign meaningful grades based on that exam score. If you know about the exam, you can assign meaningful grades based on exam scores, but not according to any optimal distribution - you would assign scores based on how much of the exam students are expected to know to demonstrate various levels of mastery, not by mathematically shaping the grades into some predetermined "optimal distribution."

Edit: Suppose these test scores have only ordinal meaning. A student with a higher score can be said to have achieved better mastery of the course but having double the score does not mean that one has achieved double as much as another student.

Your edits are not going to change the answer; there's still not going to be an optimal distribution. If I have many excellent students in the class and I do an exceptionally good job teaching them, I'll give out many excellent grades, no matter how they rank with respect to one another. If all of my students are terrible and do poorly, they'll all get low grades even if one manages to get in a few more points than another (although in that case, I'll also take a closer look at the class to see why students are doing so poorly.) If half of my class excels and the other half fails to meet minimum standards, I'll give out 50% top grades and 50% failing grades. If their abilities happen to be normally distributed, their grades will be as well. You get the idea.

The distribution of student grades should follow the distribution of demonstrated achievements. Any grade distribution that doesn't is definitely not optimal.

• Sadler, D. Royce. "Interpretations of criteria‐based assessment and grading in higher education." Assessment & Evaluation in Higher Education 30.2 (2005): 175-194.
• Aviles, Christopher B. "Grading with norm-referenced or criterion-referenced measurements: to curve or not to curve, that is the question." Social work education 20.5 (2001): 603-608.
• Rose, Leslie. "Norm-referenced grading in the age of carnegie: why criteria-referenced grading is more consistent with current trends in legal education and how legal writing can lead the way." Journal of the Legal Writing Institute 17 (2011): 123.
• Guskey, Thomas R. "Grading policies that work against standards... and how to fix them." NASSP Bulletin 84.620 (2000): 20-29.

These go into some more detail about problems with norm-referenced grading. You asked for a grade distribution that is "informative about student's grasp of the material." The literature I have cited explains that pure norm-referenced grading is not informative about that; it can only inform about student's relative grasp of the material, compared to others in the same group who have taken the same exam. In other words (emphasis mine):

It can be useful for selective purposes (e.g. for the distribution of a scholarship to the 5 best students, or extra tuition to the 5 which are struggling most), but gives little information about the actual abilities of the candidates.

Source: McAlpine, Mhairi. Principles of assessment. CAA Centre, University of Luton, 2002.

In the middle of the 1990s, the Swedish secondary school grading system was changed from a norm-referenced to a criterion-referenced system. Thus it became possible to compare (for the same population) the ability of a norm-referenced grading system and a criterion-referenced grading system to predict academic success.

The first paper looking at this Swedish data set is not written in English. (Cliffordson, C. (2004). De målrelaterade gymnasiebetygens prognosförmåga. [The Predictive Validity of Goal-Related Grades from Upper Secondary School]. Pedagogisk Forskning i Sverige, 9(2), 129-140.) However, in a later paper, the author describes those results as follows:

Cliffordson (2004b) showed in a study of 1st-year achievement in the Master of Science programs in Engineering that the predictive validity of CRIT-GPA was somewhat higher than it was for NORM-GPA.

In that later study, Cliffordson found (consistent with the earlier study)

a somewhat higher prognostic validity for CRIT-GPA than for NORM-GPA.

and that across a variety of disciplines,

Despite differences in both design and purpose, the predictive efficacy of criterion-referenced grading is at least as good, or indeed is somewhat better, than that of norm-referenced grades.

For details, see:

Cliffordson, Christina. "Differential prediction of study success across academic programs in the Swedish context: The validity of grades and tests as selection instruments for higher education." Educational Assessment 13.1 (2008): 56-75.

• @HRSE Without knowing about the exam and what it measures, you can't assign meaningful grades. If you know about the exam, you can assign meaningful grades based on exam scores, but not according to any optimal distribution - that is, you would assign scores based on how much of the exam students are expected to know to demonstrate various levels of mastery, not by mathematical tricks. Jan 4, 2016 at 6:31
• @HRSE Yet you are willing to go so far as to say that the levels of achievement of any group of students, at any kind of institution of higher education, in any kind of class, should be identically distributed to that of any other group? That seems like a much bigger leap to me :) Jan 4, 2016 at 8:22
• @HRSE You could make my examples less "extreme" and they'd still apply. It's just easier to write "all excellent students" and "all poor students" than to list hundreds of hypothetical students and their abilities. The point is that different classes are different, different groups of students are different, and there's no reason to expect any two groups to have the same distribution. Jan 4, 2016 at 8:49
• @JeffE I think the point was that it's feasible for a course to have all A's and B's and that's one example of a scenario where this would not be unlikely, not that that's always going to be the grade distribution in an advanced class. Jan 4, 2016 at 17:28
• +1 for "The distribution of student grades should follow the distribution of demonstrated achievements. Any grade distribution that doesn't is definitely not optimal." Jan 6, 2016 at 3:06

Without trying to evade your question, it is extremely difficult to quantify the impact of different grading distributions. This makes it not possible to find an optimal solution. You state 2 goals which makes optimization difficult. Further, your goals are not well defined.

On the surface, the goal of motivating students seems laudable, but we need to be careful about what we are motivating. We do not want to motivate students to strive for higher grades (i.e., grade grubbing). Nor do we want to promote a cutthroat environment where students attempt to increase their grade, by sabotaging fellow students. We want to motivate students, at both the individual and group levels, to increase their understanding. Education research is pretty clear that external motivators, like grades, do not promote the type of learning teachers strive for.

Your second goal is for the grades to be informative. But you do not define whom you are trying to inform. Do we want grades to inform students about their level of understanding, or do we want the grades to inform potential employers. From my understanding of education theory, summative assessments are not part of the learning process while formative assessments are. Further, it is not clear if letter/numerical grades are useful for formative assessments.

• My (anecdotal) experience is that having all marks high does not provide a helpful incentive, because it makes the cost of errors very high. There needs to be space for students to learn things over time and from the assessment process (except of course those assessments at the very end of the course) and benefit from doing so. Jan 4, 2016 at 7:43
• In addition to "who," it also matters "what" information you are trying to convey. Do you expect people to use the grade to estimate the student's ability to do a job, to estimate the student's ability to do well in the next course in the sequence, to select the 10 best-performing students for an award, to identify which students should be eligible for TA jobs the next time the course is offered, etc. Jan 6, 2016 at 23:29
• No offense but I have no idea how this even comes close to answering the question. Yes it is great for a teacher to use feedback from students. Are you saying that teachers should teach without grades? The answer is so vague that your answer could mean anything. Just wondering what you answer is and why the OP gave this the checkmark - I feel like I am missing something. Jan 13, 2016 at 23:27
• @blankip I am saying that the problem is ill posed and hence there is no single optimal solution. Jan 14, 2016 at 1:02
• I think there probably is a baseline method for getting an optimal grading system in almost all cases. I have done LMSs for 15 years and I have seen some really really good grading systems for very complex things. The good ones are a bit fluid and sometimes complex. I have seen step tests where they have people answer questions at a certain level of proficiency that is incredibly accurate but really time intensive to set up. Also there is nothing better than fear to motivate a human. Jan 14, 2016 at 2:31

The optimal grading distribution looks exactly like the distribution of capability in the field that the members of your class have.

Now, that's not really something you can easily or perfectly measure, but that's your target.

If you could follow your students and grade their performance in later life at a few milestones, you could conceivably estimate the information snr in your grading. That's where you'd apply those concepts meaningfully.

That has nothing to do with the shape of the distribution of grades, however.

You really can't assume any specific shape as valid or not unless you know the distribution of the population.

• Oh, I agree. I should reword that. Jan 5, 2016 at 6:31
• +1. Although the very last sentence is, sorry, simply wrong. There is no reason whatsoever why a large sample should from an unspecified distribution should turn into a normal distribution. If you are thinking of standard statistical results that imply asymptotic normality (Central Limit Theorems etc.), these involve averages of large numbers of data points, so they are not applicable here. Jan 5, 2016 at 9:07
• No, you're right. I missed the wording badly, I just have not taken the time to fix it. Jan 5, 2016 at 9:15
• I dropped my mangled attempt. I was trying to explain that you need knowledge of the population to make claims about any expected sampled distribution. Better off without it, really. Maybe I'll come up with an improvement and put it back. Jan 5, 2016 at 9:25

Years ago I was involved in the grading of entrance exams for a well-known university. Our goal, in grading, was to provide the maximum differentiation: as the purpose of the exam was to find the top candidates, whilst recognizing that for some the physics portion of the exam would only carry limited weight, I was asked to ensure that my grading was sufficiently lenient (this was a VERY hard exam) that the median would be 20/40; this required giving partial credit to partially correct answers, and did in fact produce scores from 2 to 39. Yes, that one paper scoring 39/40 on a really hard exam is something I remember 30 years later...

My point is this: "grades" can mean different things to different target groups. Questions one might ask:

1. Is the student sufficiently prepared to take the "next" course?
2. Does the student merit special recognition (scholarships etc)?
3. Does the student qualify for some "limited access" position (a job, entrance into a graduate program, etc...)

The first point should be "objective", without regard for distribution. That is, an exam should be constructed to test the necessary knowledge, and whether the person has it or not should be independent of the grades of all other students. Studying the distribution is possible helpful to the instructor (are you making "good" exams?) - but ultimately asking 10 hard questions, getting on average 4 good answers, and deciding to set the pass grade at 4+ does not guarantee that people who pass the exam have mastered the material.

The second point relies less on a distribution, and more on a "cutoff". Perhaps there are 5 scholarships: you award them to the top 5 scoring individuals. If you aggregate over multiple exams, you take their rankings on all exams (perhaps eliminating N outliers) to come up with a final ranking.

The final point is the only one where a distribution might be helpful.

Unfortunately, most grading schemes don't follow anything like the common sense approach that I describe...

• +1 this is exactly right. The OP asked about a grading system that is "informative about student's grasp of the material," which seems like an absolute (not relative) metric. Given the scenario the OP describes, it doesn't seem that a grading system referenced to any kind of distribution can convey that information. Jan 6, 2016 at 23:41
• This is a great observation that different testing objectives can have different goals for differentiation. I bet many of us read a question like this assuming our own customary use-case. The OP should clarify the answer by (among other things) explicating their use-case. Jan 7, 2016 at 6:13

Edwards Deming, a guru in the quality movement, thought that there should be only three outcomes: "Student masters the material", which is the objective of the course and the main responsibility of the instructor, "Student shows exceptional proficiency" (those he regarded as outliers and would hold out as exceptional achievements), and "Student did not master the material" (outliers at the other end). To students in the first group he would give an A (that would be a large majority), students in the second group would get an A+, and students in the third group would get an F.

• University re-labels these B, A and C. "Deming said, “Learning is not compulsory... neither is survival.”
– user28174
Jan 4, 2016 at 16:33
• One problem with that (among others) is that if these are the only official records (3 distinct points), statistical after-analysis is weakened for any purpose. For example: andrewgelman.com/2012/11/12/… Jan 4, 2016 at 18:53
• What kind of statistical analysis are you after, other than percentage of students passing, etc? The original question was putting the cart before the horse, assuming that there is an optimal grade distribution in the first place. It is not clear to me (or, it seems, to ff524 or StrongBad) that there is. The OP's question reminds me a bit of the optimal length of a man's legs. Abe Lincoln's answer? "Long enough to reach the ground". Jan 4, 2016 at 21:54
• For example: Regressions, correlations, and predictions between prerequisite courses, post-requisite courses, special assistance programs and tutoring, program exit qualifications, follow-up results like graduate or job placement, etc., etc. Anything possibly investigated in program assessment or scholarship of teaching and learning. Jan 5, 2016 at 2:18

However, it can be useful to test your assumptions by formulating your expectations before inspecting the actual statistics. What should the overall distribution look like? What distribution should which individual problems exhibit? Should a certain subset of the problems be accessible to almost all students? How many of the problems do you expect a strong, average, weak student to be able to attempt in the alloted time? Do results on some problems correlate more strongly than they should, i.e. do they really test different skills?

Then you can check if the data fit your expectations. If they do not, you did something wrong. Here are two examples.

### Multimodal Distributions

Experience suggests that you probably get something like a Bell curve if there is nothing out of the ordinary going on. In other words, if you do not get such a curve further inquiry may be warranted¹. As a specific example, if you get

you want to look for a criterion that separates the participants into two (or more) groups that explain the two bumps:

Possible criteria (which you can likely check) are course of study, participation in exercises (or some other course-related activity), gender, and so on. You can then take measures to ensure your course works equally well for everybody.

### Hardness of a problem is off

What you expect to see is this:

That is, most students solve (very) easy problems well (these should form the baseline for passing the exam, if that), average problems distribute normally (these determine everything between passing and B), and few students solve the hard problems (those that do probably get an A).

If you see big derivations from your expectations, you may have missed the mark when formulating a problem (which can influence your grading decisions) and/or your course did not promote the necessary skills as you thought it would.

1. The other direction does not work, i.e. there may be issues you do not see in a histogram.
• May I ask an irrelevant question: how were these plots generated? I like them very much. Dec 5, 2017 at 23:37
• @LeeDavidChungLin IIRC, drawn on my Galaxy Note 10.1 with Squid. Dec 6, 2017 at 9:24
• oh how did I never learn about this great app before, ha. Thank you. Dec 6, 2017 at 11:44

Some sort of Bell Curve (example #1)

This is clearly flawed because it poses no distinction of the mastery of a course nor does it show the success or flaws of the professor. If you were a really good teacher and had really good students a "bell curve" approach would mean that students that actually had a really good grasp of the subject material could get an F.

Let's just say it is a rather simple class and you are an incredible teacher. The grades on a percentage basis are 88-100%. Do the 88% students get Fs, the 90% students get Ds, and so on? And if so why would anyone with any brains take a class like this? I would have to rely on others being dumber, to get a better grade? I would think what if everyone got a 100% except for me and I got a 97%? It makes no sense. (And I know I am being arbitrary using %s as a lot of classes aren't graded like this but just using this for an easy example.

Some sort of Bell Curve (example #2)

Well same situation but this time the class isn't a "normal" class. It is at a university that has 50 openings into their engineering school with 1000 kids taking classes to get in and they take a set number of classes. Well then we might want to grade using some sort of bell distribution because it is comparing the students, not their mastery of the course. After doing this for a few courses you could probably weed out to around 50 students.

Other distribution models

The problem with stating that given a large population that you should have a certain amount of certain grades is really - as mentioned above - about comparing the students to each other and has very little to do with their knowledge of the material of the course.

Even with a simple curve model where you give the best student a 100% and grade from there you could run into some major issues while having a poor rendering of grades. If you were a terrible teacher and students had poor mastery the top student might have a 80%. Do you just bump everyone by 20% and feel good about it?

Also when doing things like that how is this reflected in your grading strategy? For instance what if the first half of your tests are relatively easy and every performs at A level. Then the next half is very very hard. The person getting an 80%, gets bumped to 100%. But then the person who got the first half all right but then missed everything on the harder material... they still get a C?

What do you do?

If your course is part of a core piece of learning and taken by most of the student population you want to work with your school on what they expect their students to get. Possibly look at the historical records for like classes. I am sure an Intro to English class has different scores at upper-tier universities compared to state colleges.

From there set objectives for the students and let them know what is expected. If everyone does what is expected and masters the material, they all get As. If no one shows up for your classes and learns nothing, they all get Fs. The hard part is adjusting the class if you feel that the grading scale is too easy or too hard. But there shouldn't be a fixed distribution. If we are talking English 101 at Harvard we are seeing at least 70-80% As.

For a upper level class a lot of the distribution goes out the window. If you are giving a science class in a specific area you must clearly grade according to how well the student grasped the concepts and materials. There grade should be indicative of how well they can expect to perform if they go to the next level. If you were teaching Quantum Physics and had a class of 10 that were absolutely below average would just give some As and Bs out? Then when they take Quantum Physics II the next professor is like "what the hell, these students aren't ready for this class."

• Great answer, and I totally agree. The only thing is, I fear that your examples of lunatic-grading-procedures are in fact widely applied, perhaps even in the majority of cases, and in fact highly recommended in some literature. Jan 5, 2016 at 2:29

The right distribution to use is the one that the students are used to.

I would dispute your assumption that more entropy is always better than less.

Lets imagine a student who typically receives a B grade, with grades going from A through D, and with F as well and various pluses or minuses, and there is a bell curve concentrating the class's results around a C+

They are content with a B, they are happy with a B+ and so on. They know broadly where they sit relative to their cohort.

Then, Professor Entropy decides to give letters all the way through W, with F still being a fail (F is the lowest grade with W-- being the 2nd lowest) and +/- still being used. The bell curving has been replaced with a flat distribution. The student receives a H+ for their work. This has a lot more entropy, but tells the student almost nothing about how well they did compared to how well they should have done, or how they did in their other subjects (C+, B- and A- using the old system). If the student wants to compare their mark to other marks they have received, they will have to convert the mark you have given them into the same distribution they are used to.

You want to compare the students to each other. They want to compare themselves to a theoretical self that studied a bit harder, or DID make time to go to that party, or moved back to subject ABC instead of XYZ where they recently switched to.

Are you the target audience for the grades, or are the students??

Apologies for lack of citation, but this is as much disputing your premise/assumptions as it is an answer.

• I dispute the OP's assumptions too, but I also disagree with this answer. Presumably if Professor Entropy explains to the students how the grades are computed, then they will be as useful to students as they are to anyone else. (Whether they are useful to anyone, students or not, is of course an entirely different matter.) It's also not clear that helping the student "to compare their mark to other marks they have received" is an intended use of grades - I don't think that always needs to be a consideration in determining what is a "good" grading system. Jan 6, 2016 at 23:35

My undergraduate course tutor's research specialism was statistical mechanics. He wrote the "grade normalisation" software for my university course which was Physics. It would take students raw trades and spit out the final UK grade (1st, Upper 2nd, Lower 2nd, 3rd, Fail). He told me that the idealised distribution should be a normal distribution; students would be randomised around a mean with more or less ability, applying more or less effort, with more or less distractions of being undergrads (aka over partying), and as those factors were random [*].

My course tutor lamented that the problem was that the actual grade distributions on typical exams was more like an inverted normal distribution; bi-modal with a group of folks flunking, a group of folks acing the exams, and very few people doing "average". This was largely because exams are not perfect and hard to pitch correctly; too hard and folks flunk and too easy and too many folks ace them. So the purpose of the software he wrote was to try to renormalise the individual exams against the student population sitting them. Rather than perfect exams grading a student population the imperfect exams can be normalised against the student population. He was a very smart scientist who had been an educator for decades so I suspect that his intuition and approach was sound.

[*] Of the two brightest guys that I still know two decades later with one got a triple 1st and a college medal, the other flunked out due to over-partying; the friend who flunked his undergraduates Physics recently took a professorial role at a respectable university with a research specialism of international development.

• Regardless of how bright this tutor was in mechanics, as an education statistics professional I would call this attitude totally malformed. The goal of (STEM) courses should be to master a certain set of skills, and the bimodal distribution is shining a bright and consistent spotlight on the simple fact that some have passed and some have failed. That's not because "tests are hard to write"; that's incoherent BS. Mangling the actual data to fit some preconceived hypothesis is a kind research fraud, and he should know better. Jan 5, 2016 at 2:25
• If your typical score distribution is bimodal, then this logically cannot be due to the test being "too easy" or "too hard" - either of these would simply shift the distribution, not create multiple modes. And why "should" grades be normally distributed? There is no law of large numbers involved that I could discern. Jan 5, 2016 at 9:03

My philosophy professor's grading philosophy was that by very definition, most people are average (i.e. C)

Students asked if he graded on a curve and his response: Yes - if my grades don't fall on a bell curve with the bulk of the class making C, several D's and B's and maybe a couple of A's and F's then I've either made my tests too easy (grades skewed to the A) or too hard (grades skewed to F).

I suppose it depends on if you're trying to push a few of your students to succeed, or if you're trying to help as many students as you can get good grades.

• "then I've either made my tests too easy (grades skewed to the A) or too hard (grades skewed to F)." - Or my teaching was more/less effective this semester, or the admissions committee has changed their standards and are admitting students who are more/less prepared, or a newly available set of online lecture videos now enables students to learn the material despite my poor teaching, or the instructor of the previous course in the sequence covered a lot of "my" material, or the instructor of the previous course in the sequence introduced a misconception that I haven't corrected, or... Jan 6, 2016 at 22:02
• I actually found that I agreed with his approach - but that's probably why I was one of the 50% that didn't drop his class. I mostly made B's on my tests. I feel like I earned that grade - and I'm a lot more proud of the A that I made in that class (he offered a full letter grade of extra credit) because of the effort I put into it. I also learned a lot more about writing and arguing. The delicious irony is that I got terrible grades on my multiple choice tests in a history class. Usually because she'd ask dumb questions like, "What's the most/best/<relative>...." Jan 7, 2016 at 0:15
• I could've taken the lectures and reading and argued that any of the answers were the most or best, unless there was something that was just plain wrong. Like Abraham Lincolon as a general in the Athenian military or something to that effect. Jan 7, 2016 at 0:16
• You feel like you earned your grade because of the effort you put in, but keep in mind that with his grading strategy, you could have earned the same A if you hadn't put in effort, but the rest of the class put in even less. Jan 7, 2016 at 0:23
• This is a philosophy professor who (like many in many disciplines) doesn't understand statistics worth crap. I wonder what the response would be to point out that students are actively incentivized to sabotage other students in this system, because then the scaling will push their own grades higher for the same results? I would claim that this system is unethical in a variety of ways. Jan 7, 2016 at 6:19