Should unreadability or sloppiness of students' work be reflected in their evaluation?

Question

When evaluating and grading students' work in technical fields (like computer science or mathematics), I often face ones that are really messy, sometimes even unreadable (I mean structure, not - handwriting). It's not clear at all what part belongs to what or what is the final result. Instead of spending a lot of time trying to figure out if there is a good result buried somewhere in the mess, I'm tempted to just fail such a work as unreadable.

While these (I'd say) formal aspect don't reflect the ability of the student to solve the problem, it strongly reflects his/her ability to present the solution. And in their future professions, it won't be much good that they can solve problems, if they're not able to present the solution to someone else. Or for example grant agencies simply reject proposals that fail even slightly given formal criteria.

My questions are:

1. Should the evaluation include evaluation of readability, proper structure etc. for example in a textual form (without impacting the final grade perhaps), something like "The solution is completely unreadable." or for a larger work "The solution lacks a proper structure - there is no introduction or conclusion of what have been accomplished." etc.?
2. Should such deficiencies be also reflected in the grade? If so, how much? Would it be acceptable to even fail a work just because it's completely unreadable?

Answer 1

I think the most important part of your post is in your comment

Or for example grant agencies simply reject proposals that fail even slightly given formal criteria.

If a student isn't corrected for sloppy work while in school, when will he or she improve? The question really boils down to

How best can we as instructors give guidance to students so that they will succeed in the future?

Evaluations in an educational setting can serve many purposes, but fundamentally they should be used as a tool to help students succeed. Obviously, you have to be careful because a poor grade on a transcript can have long-term ramifications, but ignoring sloppy or unreadable work by trying to see through it is doing a disservice to the student on a particularly key issue: communication of an idea is as important as the idea itself.

To answer your questions:

1. Should the evaluation include evaluation of readability, proper structure etc...?

Absolutely. To ignore this would be bad pedagogy. Determining how to relate this to the student without causing a misunderstanding about the idea itself is the difficult part. Encouragement or direction to seek out writing help is a good idea, as is having students re-write or re-submit work that is sloppy. Hopefully the work improves throughout the course; being picky at the beginning of a course can set a good standard for the rest of the course.

2. Should such deficiencies be also reflected in the grade? If so, how much? Would it be acceptable to even fail a work just because it's completely unreadable?

The answer to this ties directly back into the first question. I would suggest letting students re-write or submit material without penalty (or with minimal penalty) early on in a course, with the understanding that they must improve by the end. If your only assessment is a final project, consider allowing a re-write with an incomplete grade. But, if that is your policy, I'd strongly suggest providing a time for draft review well before the final project is due in order to minimize the need for this route. As always, I would also spell out in a syllabus your policy and the fact that presentation is a part of the course, and I would also discuss this on the first day of class (with examples of past work that is sloppy!).

You've posed a hard question, but an extremely important one. I probably suggest rejecting 30% of the papers I review for conferences simply because they are unreadable. Most of the time I don't even get to the idea behind the work before I realize that it doesn't matter what the idea is because it is too poorly written (and this ends up in the review). I want more papers that are well presented, because I'm certain some of the papers I give poor reviews to because of this problem have great ideas behind them! Teachers that encourage students to improve their writing and presentation are necessary for this to occur.

Answer 2

While I do agree with Peter Jansson's answer that we should make it completely clear to students what they need to do to have their work understood, I feel differently about the impacting on assessment.

In industry, the ability to write readable (which means maintainable) code is critical. The ability to write a really cool, super efficient block of logic which nobody can understand is simply producing a ticking time bomb which will explode as soon as someone else goes to edit it. Students need to know that you are not just being a 'stuffly old prof' but rather that there are certain expectations from the profession and they must live up to them. Coding is not just about making a solution - it's about making a solution which will not explode 12-18 months later.

On the academic side, I also have students who have similar problems where they do not care much about being careful in their writing. I generally handle them like this:

1. If I can understand it, but it is difficult, then I will mark them based on the overall quality of their work. If it is difficult to understand then the quality of their work is not great. Even if they are far smarter than me, they should be able to write in such a way that they can be understood.

2. If I cannot understand it, then I simply fail them. They are responsible for showing me that they understand and if they do not do that, then I must fail them - this is my responsibility.

So, yes, educate them. However, I would not let poor quality slide.

I try to remind my students of the story of the family. Every happy family is happy in the same way. Every unhappy family is unhappy in a different way. The students are usually lost at first, then I explain to them that for the family to be happy they must get 100 things right. But, get anyone of them wrong and the family is unhappy. If they want to be happy then they must get it all right.

Answer 3

Proper presentation of results is a vitally important part of modern research. It's also a necessary skill for any field of work into which an academically trained individual will enter after graduation.

If one does not learn how to exchange information with other people and organizations, the results can be catastrophic. Edward Tufte's Visual Information uses the Challenger explosion as a case study in exactly what can go wrong when communication skills are ignored at a fundamental level. The launch was not aborted because the engineers just provided a bunch of disorganized data to management, who couldn't parse it out to understand that it was too cold to launch.

So, as much as possible, presentation and organization of material should be considered when assigning grades. Now there are situations when it would be unfair to penalize: for example, in the context of a timed exam, where students may feel rushed and may not be writing neatly. (However, I also feel that this is a largely contrived circumstance, which doesn't reflect true understanding in many important ways.) In any situation where they have had time to independently prepare their work and submit it, then presentation and writing style should definitely be taken into account.

However, at the same time, unless the class is a writing class, then it's unfair to fail or strongly penalize a student who has turned in work that would otherwise be satisfactory, but was not put together well. Personally, when I am grading project reports, I "control" for this by making a decision on what the grade for the technical content of the project. I then give a bonus "partial" grade level for a well-organized and well-written report, and similarly deduct a "partial" grade if it's substandard. So, for instance a report that would have gotten a B+ could become either an A- or a B.

Answer 4

When I set marking rubrics for my students' work, I usually include two main areas: Clarity and Contribution.

Clarity specifically addresses whether the student has communicated their ideas (or answered the questions) in a clear and unambiguous manner. As such, poor presentation or (really) bad grammar impacts this portion of the mark.

I generally don't detract from the student mark for speeling errurs [sic]; even though I am a native English speaker, I am an Australian working in the US and am frequently in a quandary over whether to use Australian or American spelling.

My present students are generally from non-English speaking backgrounds, and I find their clarity generally very good, even though their turns of phrase sometimes seem awkward to me. I give feedback on this awkwardness but, unless it impacts the clarity of their presentation, I don't dock marks for it.

Contribution is used to address the correctness or fitness for purpose of the work.

I also add a further C, Collaboration, if the work is a group project.

---

To directly answer your questions: Yes, and Yes.

As a side-issue: Because many of my students are non-native English speakers, I also tend to allow double-submission of assignments. The students may submit their assignments twice before the due date.

I give an undertaking that I will do my best to give early feedback on an early submission (because I am an adjunct, I make it clear that "early" means the submission has to happen before a weekend before the due date). The students can then take my feedback and implement it on their new submission. I find this improves the quality of the presentation (clarity), but not so much on the contribution (unless they were really, really out of line on the first submission).

Unless the students do a complete re-write, the second marking is usually just a "delta" / "diff" on the first.

Answer 5

Ideally, no. The main problem when evaluating is expressed by a saying in my native language "muddy writing, muddy thinking". Essentially, if you cannot decipher what is written, either because it is sloppy writing or poorly expressed, then one faces problems when grading. Students often complain about grading where they sense they are misinterpreted etc. This is when we cannot understand what they are writing. Often, I believe, we also give people the benefit of the doubt and grade more leniently than what the verbatim answer would state. This is, of course, never picked up by a student. Hence a catch 22.

What can we do? The problem lies in the examination.

First, we can provide clear guidelines about clarity of language and writing. This should be pointed out long before the exam, in fact early in their education. A study of Swedish students showed that they learn two ways of writing in pre-university levels. In languages, focus is on grammar, spelling and language but not content; whereas in other subjects focus is on content and not language. When they arrive at the university they do not realize that both are important; we need to tell them.

Second, we can chose other means of examination where those with poor hand-writing can do better. Examples can be many: from essays through take homes to verbal and of course highly depending on the course, contents, level etc. Examples of the means of grading is to have external graders (more resources!) or to do as one of my teachers did; ask students to request everyone writes the exam with the pencil, and then after the exam use a pen to add corrections to their own exame as the teacher went through the exam. The exam was then collected and graded by the teacher. I mention this as an example, fully realizing it would not work in every situation.

So the answer for me is no, but our assessment is always tainted by many factors whether we want it or not. To reduce the element of subjectivity is what we would be looking for.

Answer 6

Beauty is in the eye of the beholder.

You as an instructor may see hundreds upon hundreds of papers a term. The students do not have that same luxury for comparison. If penmanship and other elements of style are important to you as an instructor, then as a student, I suggest you express your standards and expectations up front, even if it seems that is something I should know and do coming into your class.

I expect no less from an employer as an employee.

1. Note the degree of impact for presentation on the syllabus, even if there is no plus points for exemplary, and only negative points awarded for substandard.
2. Provide a sanitized modicum of 'bad', 'passable', and 'exemplary' presentation for comparison.
3. Encourage them to seek clarification or special consideration up front, as opposed to seeking remedy to penalties after the fact. They may have certain existing conditions which precludes them from generally meeting those standards. Or more commonly, they may not have been exposed or acclimated to the technical writing style you are accustomed to in that particular field..

Answer 7

I particularly like the answer that suggested having rubrics to identify criteria such as clarity.

When it comes to a situation where work is completely unreadable, I apply the pedagogical pattern known as Grade it again, Sam. Very often, the student who submits unreadable work is simply not making an effort. When I refuse the work using this pattern and give the student a second chance to improve readability, the second work is always better and subsequent works are usually also better.

Answer 8

As a current student of computer science (master level) I can say that commenting on the readability of work from students, providing pointers and even taking it into account when grading is of great importance. It will benefit the student a lot in the long run. At my university, many professors point out bad readability and incoherent structure, although mostly on project/paper type assignments and less on weekly homework assignments. However, it is considered important in all cases. I'm now writing my thesis and have written two research papers as part of research projects (for publication) and the comments I got on structure and readability greatly improved my writing.

Also, I see from fellow students that they write solutions without providing reasoning and without introducing the topic and the problem they are solving. I believe this is very bad style (for group work this has unfortunately led to me rewriting the group documents many times simply to improve the structure). However, students don't often see a problem because it gets accepted by the teacher.

An important note though is that I think grammar and spelling should be of significantly less importance. For instance, the Chinese students in our class often forget the word "the" in writing (all documents are written in English for my master). Although this affects readability and is often pointed out by teachers, it is generally not considered an error significant enough to deduct points.

The line between decreased readability due to grammar/spelling and actual incomprehensibility is hard to determine, and should be determined on a case by case basis. Still it is important that students also learn this.

In short, definitely help students in improving their structure and on communicating their ideas and force students to do so by making it affect the grade! It will only help them like it helped me!

EDIT: As a side note, handwritten solutions are generally not accepted where I study, but if YOU are requesting handwritten solutions, it would be unfair to deduct points for bad handwriting. Some people just don't have a nice handwriting, an unfortunate result of the modern computer era. If students CAN but don't have to hand in handwritten solutions, then it should affect the grade as it is the students choice to write instead of type (if it significantly affects the readability of course).

Answer 9

What is your evaluation measuring? Are you grading a 2nd grade math test, or are you evaluating a doctoral thesis on neuroscience?

If it's the former, then no; sloppiness and structure take a backseat to getting the subject matter correct. The reason is your focus should be on evaluating the crucial element of the subject being evaluated. At early levels of development, there must be some leniency given to presentation in order to focus on the fundamentals.

Of course, there is a reasonable limit to how sloppy or unreadable something can be before the evaluation ceases to be useful. Intentionally sloppy responses relying on confusion in the hopes of gaining partial credit don't deserve any credit at all.

If you're evaluating a student in a near-professional setting, e.g. university and higher, then it makes sense to consider presentation as part of the grade. Well structured, carefully written responses should get more credit than sloppy and careless ones. Presentation becomes important because in the "real world" people evaluate you based on the whole package. Nobody is going to buy a bottle of Tylenol with a hand-scribbled, barely legible label. Likewise, even superficial details should be considered important in high level evaluations.

I was a sloppy student through primary school and even most of high school. Many teachers simply refused to grade some papers that were unacceptably sloppy, and others graded what they could within reason. None of those teachers who refused to grade a paper stick out in my mind, nor did their insistence on neatness change my behavior. I excelled in the subjects where my understanding of the core materials mattered most, and did not focus my attention on subjects where presentation was considered more important than the concepts being taught.

Overall, I graduated in the top 5% of my high school and went on to a good college. By the time I got my masters degree, my value of structure and presentation has improved sufficiently that I can survive in a professional environment. Becoming a professional, I believe, is the goal of "teaching" organizational skills.

Answer 10

When you discuss formatting constraints of the grant application process, you're getting close to a solution, I think. There are really two components here; one of these is structural/formatting criteria, and the other is clarity. When you're in a situation where structural requirements and great care are part of the lesson you're teaching, I find it best to refuse to grade work until it has been presented in the proper format, and apply whatever late-work structure you normally use.

This teaches the lesson that formatting is not optional fluff, but is instead a precondition to being taken seriously. That said, if you're going to follow this method, your instructions on format should themselves be crystal clear and unambiguous. Ask a competent peer to follow your formatting instructions and see if they can arrive at the right format.

The second question about messiness is more troublesome, and to some extent your response should depend on where you're situated within the broader track your students will follow, and what constitutes messy/sloppy. I think your response should be two-pronged:

1. I wouldn't advocate failing students outright for unreadable work, but let them know you can't grade the assignment until you can understand it, and ask them to discuss their work with you one-on-one. Like above, allow your late-work policy to degrade the value of their work if they don't come address it swiftly. I ran into a few students who had a sense of what academic writing should sound like, with the result that they were completely incomprehensible when trying to parrot a voice they had no real command of. But they were able to explain it in person. This method allows you to show how imperative clear communication is--it's a precondition to being graded--and avoid invalidating what may have otherwise been good work. The ultimate penalty is based on how quickly the students resolve the problems with their work.

2. Include some metric of style/quality/organization on your rubric. While I understand the inclination to see writing as something students should already know how to do--and that you probably don't see yourself as an instructor of writing--I think rubrics should always always always reflect the qualities you actually expect of good work in your field. Your rubric telegraphs your own priorities, and if you don't communicate clarity and style as a concern, it shouldn't come as a surprise when students think it doesn't matter.

Answer 11

I am not a teacher, but I am in my second year of my associates working toward my bachelors. My opinion is that yes you should, it will only hurt them later in life. In high school I passed my classes with C or a C+, now in college I am getting As and Bs. The reason is not that I am magically smarter, it is that I apply myself at least twice as hard.
There is so much information out there to learn anything, APA or MLA, and sites like Grammar Girl and reference generators. I think spelling should and grammar should be graded hard, maybe not at the first of the semester.
If they are bad at first, correct them and give them places to go for examples of the correct way(s) to do it. If by mid-term they have not figured it out then grade them hard. All work should flow; if it does not, let them know now because in their career they will not get a nice correction and time to fix it. I work my butt off in school and feel as if I get graded hard, but it is when the teacher gives good feedback that I learn and make the change. So giving them a grade lower than an A needs an explanation of what they need to fix.
This is why we are seeing on job descriptions -Bachelors from a reputable University- teachers need to be hard so that the student walks away with a real education not just a piece of paper that cost them thousands of dollars.