Is there a way to stop students using AI to generate essays?

Question

A friend of mine teaches German at a university outside the EU and has observed a rise of the quality of essays written by students, comparing the same years of study: first-year students now compared to first-year students a year ago, second-year students now compared to second-year students a year ago, and so on. A year ago, the typical essay contained a number of language and stylistic errors, weird word choices, and awkward non-idiomatic expressions. Now, some essays read flawlessly - but somewhat blandly and emotionlessly, not really human.

So, she's afraid that some students might be actively using AI to generate essays, and she is stuck as to what to do about it. Even when her gut tells her that an essay is AI-generated, she needs some sort of proof in order to take an action about it. A baseless accusation could easily cost her a job. At the same time, her conscience doesn't let her just sit on her hands. Also, one of her KPIs is the average student score for the final exam - and the average score will surely go down if she doesn't find a way to force her students to write essays on their own.

So, she asked for my advice, and I'm typing this question here: Are there any strategies of building a proof that an essay is AI-generated?

There are various AI-detection tools on the Internet, but they are mostly for the English language, and I'm unsure how good are those that check German texts. I selected a random German language AI detector and tested it with a text generated by ChatGPT, only to be disappointed when the message indicated that the text appeared to be written by a human. Another German AI detector returned an AI-probability of 60% for a text actually written in the pre-AI era.

I'm seeking not only suggestions for reliable AI detectors for the German language, but also general advice about how to deal with the problem in a university setting.

Answer 1

AI tools such as Chat GPT are really helpful for language learners, in that they can quite reliably pick up grammatical errors and other problems, and (albeit with somewhat less accuracy) can even explain what the problem was and how to do it properly.

Rather than banning such tools, I would suggest requiring them. Have students turn in:

• The original (non-AI) version,
• The corrected (post-AI) version, and
• Some commentary on what mistakes the AI caught and especially anything the AI flagged but the student isn’t sure about

That draws a nice clear line between legitimate and illegitimate use of resources. Some students may still try to cheat by generating the original essay, inserting plausible errors, and then “correcting” the errors, but this seems like almost as much work as doing it legitimately.

Answer 2

Here's how I have been dealing with it: make it harder for the students to cheat than not cheat. You can't stop them from cheating, but you can definitely tilt the balance to make cheating less attractive. Some strategies that have worked for me:

1. Make the instructions so specific so that they are hard to enter as a query on a LLM. Add very specific formatting instructions, and specific instructions about the structure of the paper. For example, require that the first paragraph needs to address X and Y subjects, using the funnel writing style. What's the funnel writing style? Read the textbook.
2. Split papers into small submissions, giving the instructions for the next part one week at a time. It's hard for students to query LLMs so that their following submission matches something they submitted a week ago.
3. Have students write their papers in a shared Google Doc and disallow copy-pasting from another document. By "disallowing" I mean that they can't just claim they wrote the paper in another program and then just pasted the whole thing into Google Docs. They can still copy/paste as part of the normal editing process. Students must write their drafts and make all edits directly in the document while preserving the document's history. The downside of this is the realization of how little time students actually spend doing homework.
4. Make students write short paragraphs while in the classroom, and tell them you will compare writing styles against their final submissions.
5. Reason with your students. I often ask them: "Would you pay someone to go to the gym for you?" The whole purpose of the exercise is to force them to go through the difficult process or organizing their thoughts in a document.
6. Grade fairly. I often establish a baseline at the start of the semester (e.g. make them produce a writing sample) and tell students that I will grade them not against each other, but against their own baseline. It lowers the anxiety of having to produce a perfect piece of writing, when no such thing exists.

None of these strategies will stop a student who is determined to cheat, but they have helped me lower the amount of essays I see obviously composed by LLMs.

Answer 3

Frame challenge: Is there a way to stop students from using a calculator in maths homework?

It seems like the humanities are now encountering the problem which maths teachers faced when calculators became a thing. I remember my teachers saying: You should learn doing calculations in your head - you won't always have a calculator with you doing the computations for you (and I finished school in 2015!). And look where we are now.

Do you really want to be similar to those guys?

The availability of calculators meant, that the requirements for a mathematician changed. You now do not have to be any good at computation, because you can make a computer do it. So you can focus on the modelling.

Given that your students will have language models available for the rest of their lives, ask yourself the question: What can they do which the language model can not do? Grade them on that.

If the language model could actually do anything that is required from a student in your subject (which I don't actually believe), then your subject would be obsolete. Why hire anyone from that subject if a language model does it better? So focus on the things language models can not do, grade only those things and let the students use LLMs for the rest. Because that is actually preparing them for the future, which is your job after all.

Answer 4

I think the common consensus is that, when properly used, AI is near impossible to detect. So the best you can do it make it so hard to use properly that most people won't bother.

If the main goal is language instruction, then the precise topics of the essays should be somewhat flexible. So I would make use of that and pick things the AI cannot know about. E.g. have the students discuss recent news events past the AI cutoff. Or in a beginner class on directions, have them describe in detail the way from the building entrance to the classroom.

You can also add in additional constraints that should confuse an AI. In a language class it should be perfectly fine to ask them something like to use precisely 10 words from a given 20 word list and use each of those words precisely once. This is a nice exercise in vocabulary, but from what I can gather something that LLMs are notoriously bad at. In particular since you ask about German which has different conjugated forms of the same word.

Answer 5

Check the sources

One of the known problems with generative AI is that they make up everything, often including reasonable fake citations. If you set limits on acceptable citations so that they can be tracked down using library resources for example and actually check the citations are valid and relevant you should be able to catch papers that are completely AI instead of researched.

Answer 6

This might seem far fetched, but Honor Codes have been fairly successful in the past to avoid some practices that students might use to avoid learning and cheat for grades in various ways.

An Honor Code needs to be clearly stated and both the reasons for it and the consequences of not adhering to it need to be part of that statement. The main reason, of course, is to try to get the student to accept the fact that hard work is essential for true learning. In some places, breaking the code has severe consequences, up to expulsion.

This won't, of course, end all cheating, and it also requires a high level of instruction and reasonable expectations of students.

But, in the presence of an Honor Code, students could be told of how it relates to using AI shortcuts generally and how it impairs true learning.

If the students are generally "learning driven" rather than "grade driven" then an Honor Code can work in the main. And that, in turn, implies that grading be of an individual and their work and not competitive in any sense. If you don't introduce adverse incentives, things generally work out better.

There is unlikely to ever be a perfect solution to the problem of cheating, of course.

Answer 7

Make it hard for them to use AI.

What some teachers have been doing is giving the writing prompt in a Word document and hiding a secret prompt in the text by making the font on it white. They use something small and highly unrelated in the prompt like, "mention maraschino cherries in the paper." The student will not see the secret prompt, but AI will when the student copies and pastes it. Any paper containing something about maraschino cherries is likely guilty of using AI.

Once a student is caught maybe they will be embarrassed enough not to do it again.

Teach them AI can be a resource

Another way to handle it is to require them to use AI as a resource. Have the students learn that it's a tertiary aid at best. They can pick apart what AI has given them, hunt down sources, reword, discard, etc... Then you might get yourself a resourceful bunch of kids who have learned how to create something unique despite the increasing reliance on AI.

Answer 8

I've responded by 1) requiring disclosure, 2) potentially requiring discussion, and 3) tweaking questions.

I've deliberately avoided forbidding use or creating a slalom of ad-hoc requirements to discourage or detect it. It's part of the landscape, and while all of us, students and I included, are still learning how to use it well, I feel it inappropriate to reject AI use outright. I just seek to remove the temptation for students to outsource thinking to it, i.e., as a substitute, rather than complement, to learning.

As context, I'm an industry-experienced adjunct lecturer in an interdisciplinary field with a lot of autonomy to set my own policies.

1. I require disclosure by asking students to specify when submitting their writing if and how they used AI tools while writing; and if they did, to briefly indicate challenges/issues they faced. I explain in class that this is both in the interests of fairness, but also since I want us to be able to have a discussion collectively what challenges and issues people found (though I think I'm not actually going to have time to do that, since I'm behind....)

2. I added a clause to my syllabus that I can require any student to come in and discuss their paper viva voce prior to receiving a grade. I've explained orally I will not do this generally, only if they self-report extensive use of AI or I suspect it, i.e. something feels off. The goal of the discussion will not be to "catch them out" on using AI, but to ensure they understand and stand behind what they wrote, regardless of any AI assistance. I therefore also emphasize they are accountable for the paper they submit, that they can't blame AI for any inadequacies. I'm going to have that discussion with one paper where the student has self-reported extensive use of AI since I'm curious AND want to make sure they truly own the outcome, and with one other which seems AI-ey in style (to me subjectively) even though not disclosed (there may be a language comprehension/cultural issue too).

3. I've ran my questions through chatGPT and claude.ai to see what the "AI standard" is. In most cases it's just not too good and students obviously need to do better. Perhaps more careful AI prompting will help, which I don't mind if students disclose. In one case the AI answer was too good; I've changed the question since in retrospect it was too easy anyway, it was just structured regurgitation. My fault as a pedagogue that I didn't previously notice.

Answer 9

Block the website

Have the students write the essay in class, and give a list of what resources are allowed. In particular, note that the use of generative AI is excluded.

This is similar to when an assignment in math class needs to be calculator-free: you need to do it in class to make sure no calculator is used. In fact, I would recommend getting advice from math teachers.

If your friend has no way of blocking individual websites, a crude, annoying, yet effective solution is to ban all computing devices (including phones). You can't run a LLM in your head!

A not 100% effective but less annoying solution is to let students do the assignment at home, but they must record themselves doing it. Your friend would then randomly sample footage or use an AI to check if they are doing the assignment legitimately. It's possible in theory to cheat this with video editing or generation, but it would be massively more effort to pull-off. And random sampling should be a sufficient deterence.

Answer 10

Felix's answer is already excellent. And I guess I'm making the same point but phrased differently. You are supposed to be teaching people skills that are useful. To be useful you have to be able to do something that a computer can't do cheaper and faster than you. If you are doing something by hand that a computer can do better/cheaper than that is neither a useful academic pursuit nor is it going to land you a job. That's okay for a hobby. But university isn't really about teaching you a hobby.

So the answer is to not care about whether the students used LLM's. You care whether they can complete the assignment. If LLM's are useful to the students they should be using them. (It is certainly what their future employer would want them to do. They care about the output not how it got there.) If that allows you to now move the bar higher and set higher demands and more difficult problems that's great!

Of course, using LLM's can have downsides. And in those cases where LLM assisted work is worse than independently produced work you should let that be reflected in their grade. Just like their old mistakes were reflected in the grade.

I'm sure there are many tasks you could give the students for which current LLM's aren't going to help much. Give some of those problems. It will teach them that they have to understand things for themselves. Not because it is some weird arbitrary rule made up by a professor but because there are problems they cannot solve otherwise.

---

Some specifics since this concerns a language course:

1. Professional translator commonly use LLM's to assist their work these days. It is good for students to be familiar with such a workflow.
2. One goal of language courses is often to have rapid on the fly conversational skills. Clearly LLM's (currently) won't help with that. So oral exams would provide a great (and useful!) test of the students' level without AI assistance.

Answer 11

In principle, you cannot detect AI much less prove that a text is AI. The whole point is that AI generates content that's "just like" an intelligent human's. As AI tech advances, the output will be more convincing, and it's already pretty advanced.

The best true test I can think of is something like a closed book exam. Have them write an essay in 2 hours, while being observed, with no technological aids. If they fail to produce essays similar to their homework, judge it as AI usage. But is it really worth it?

In practice, ChatGPT (and others) have some idiosyncracies because of how they are trained. These can be used to detect AI-like text. However, it will never be conclusive because there is nothing stopping a human from writing similar text "manually". Still, you might have some limited success with such tools, you could even try to ask ChatGPT itself.

The problem is that AIs advance and evolve, so these detection programs are locked in an arms race with them. At this point it's a whole ongoing project in itself to get comfortable detecting AI essays in your domain, and keeping up with the development. And all that, just to stop students from writing mediocre essays?

Now, some essays read flawlessly

So, students are using available tools to do better. Sounds like a great development.

but somewhat blandly and emotionlessly, not really human.

So the real problem is that they're bland and emotionless, not the AI. So, during the class explain what makes an essay bland and emotionless, and tell students you will grade based on that. Warn them that if they use AI, they will have to work extra hard on the bland/emotion part.

So, she's afraid that some students might be actively using AI to generate essays, and she is stuck as to what to do about it.

In brief, the solution here is probably to get over it. It's not a judgement on my part. Whether you like AI or not, it's here to stay, and it's getting better very fast. For essays, it's already excellent, and you will not put the cat back in the bag.

When calculators became commonplace, many math classes had a similar dilemma. Are the students really learning their numbers if they just punch it into a calculator? My own schooling banned calculators almost entirely, so I got very good at doing math in my head in school. As soon as I stopped being in school, and started working, I bought myself a $20 calculator (touchscreens have poor usability) and never looked back. I am still better than most I know at calculating tips and what not in my head. But I am certainly rustier than I was back in the day. And how does it affect my life? Hardly at all. I have to pay the very small calculator tax, and in exchange, I am free from tedious arithmetic and numeric errors are reduced by orders of magnitude. Honestly, all the years I spent memorizing squares so I could take square roots in my head were a waste of time.

I think the smarter math teachers responded by adapting. Since the calculator removed the challenge from a major, mechanical component of the course, they used the new space to emphasize the high level more. Since being good at actually doing addition is no longer important, they focus on the nature of addition.

Similarly, now that simply producing grammatically correct and coherent text has been trivialized, why not roll with it? What is there to gain from teaching students how to do something by hand, when after school they will inevitably use a much more efficient machine for it? Instead, you can focus on the deeper/higher questions of what makes an essay good. In fact, you might as well recommend them to use AI, and even advise on how to use it more effectively, so that they have more room to explore those finer points.

Answer 12

This is not even useful for you since you don't control the system but here's the truth.

Take away grading that has any ramifications outside of class. The point of the class is to learn. Students are afraid the grade hurts their GPA/ability to graduate the class, because the GPA matters/they want to graduate they cheat.

We begin by first considering the easy case of math where it should be

1. You go to class to learn; those grades aren't reported anywhere.

2. At the end of class you can take a written exam that is proctored by a grader and if you score sufficiently high you pass the class.

Now we go to your harder case of German it should be:

1. You go to class to learn; those grades aren't reported anywhere.

2. At the end of class you can take a written exam that includes a large handwritten essay section and perhaps an audio-based oral section.

At this point the students have no incentive to cheat. If they still do then you don't need to catch it/help them either. They aren't getting any unfair benefit for it. Making the whole class depend on 1 exam might be unfair so it could be 2,10, however many in-person exams it needs to be.

But any work that is taken home is basically fair game for the AI to cheat without being detected: if not today, then certainly another day in the future.

Answer 13

Don't stop students using AI, stop students from diminishing their own learning

I say this because you haven't actually said what exactly the issue is- the language is flawless but feels less human? How is that an issue? How does this diminish the learning progress of the student? I think the issues need to be clearly spelt out before we address them. Ideas such as having students write in a classroom in handwriting or on locked down computer systems rely heavily on room, personnel and hardware- which may not be something you can control.

I want to share my perspective here as someone who taught a course in which students have to hand in a research paper. After ChatGPT became widespread, the quality of the papers dropped immensely. Students would run an experiment, and then tell ChatGPT to write what they did, using notes from the experiment. That sounds really good, doesn't it? The student still does the research work, but can use a tool to help getting that work to paper! It was a disaster. Here is why:

1. Students would generate language beyond their own level. This meant that oftentimes, synonyms were used that were not synonyms; sentences went from short and concise to long, verbose and unnecessarily wordy, not adding any information. Students could not tell that the quality was worse, because they did not have the required skill level and expertise to tell good writing apart from bad writing.
2. Students would generate more text than necessary, often leading to incoherent writing. One thing that GAI makes very easy is to generate volume. And for many students in many school systems, volume equates to quality (whether this is actually true, is another matter- but many paper requirements have minimum lengths).
3. Students would disengage from their own written work. Even when students had clearly carried out their research, had been in contact with me for feedback, even when they were otherwise good students showing understanding of the subject matter- those that relied on GAI to have the paper written for them, rather than support themselves writing it, ended up handing in terrible papers. The quality of their research work did not translate into quality of writing, because they did not engage with the text written for them. Why would they? The use of language is higher than what they could write themselves, and there is no need to check for typos. Students have learnt how to write text, but not how to edit text. This is a skillset that they need to have to, as the AI would phrase it, "leverage the power of AI tools".

So rather than ask how to stop using something that will likely become as ubiquitous as mobile phones or the internet, ask how it harms the students. In my case, several rule changes were made:

1. Instead of setting length minimums, maximums were set- and those were tightened, too. This requires higher information density in the text. This re-engages students with the text, because they need to reduce verbosity. I expect this to become less of an issue as GAI evolves, but in scientific writing, conciseness is a marker of quality, and should therefore be rewarded.

2. Students were taught on how to use GAI. Besides text-generating GAI, there are also translation tools such as Deepl (which is also available for German) which are very good, when used well. Students are taught which type of tool is good at what, they are taught what the issues with generated text are, and what they need to do to edit it. They are shown issues with fact hallucination and the danger of asking the AI questions in a narrow field of expertise when the students themselves can't tell truth apart from hallucination. They are shown how to use the tool to help them with their own work.

3. The grading was adjusted. Grammar and spelling mistakes are now no longer graded- more emphasis is placed on the use of language to get information across. The content-relevant grading parts were adjusted so that more emphasis was placed on the coherence of the paper as a communication tool, not just the research work that was done. This meant that students who write without AI could still be wholly rewarded, and students that use AI would need to engage with the generated text to meet the requirements.

AI is here to stay. Any answers that suggest exploiting a weakness of the current AI (such as no sources beyond the training data) is bound to be out of date quickly- we want to avoid an arms race with an industry that benefits heavily from improving the GAI systems that are already in use. And in the case of source-checking, places massive additional workload on the grader.

Answer 14

My suggestion would be:

Either tell them to write an essay in a live classroom. Confiscate their cellular device and laptop before the start of the writing.

Or, eliminate essay writing altogether. Device new metrics to assess writing proficiency.

Answer 15

There's an obvious pseudo-problem when "the internet" can do much better than many college freshmen and sophomores. It's not a genuine "problem", since it is a fact, much like the points about even-very-basic calculators being able to do arithmetic better than most children, and contemporary phones/computers/whatever being able to "do" (the algorithmic parts of) calculus much better than many college kids.

To my mind, to insist that every young person in college do better than AIs is pointless and unreasonable. Um, they literally cannot!

And, dangit, if we decide to demand of college freshmen that they do things that AI's cannot... then we are... seriously ... placing a far-too-high-bar.

In the near term, "soft skills" may differentiate humans from AIs, though, well, ahem, lotta humans are worse than even the old-fashioned AI's at comforting and helping people in consultation situations.

So, seriously, I do wonder how/why we will constrain AI's. Sure, they may take away jobs from humans... but, on another hand, maybe they will benefit humans by being more competent than those humans whose jobs they've taken?

Answer 16

Have them write about things AI won't know - current events, topic the grader is very familiar with, etc. For example, I had students use ChatGPT to write about biodiversity in African countries where I lived. ChatGPT got many details wrong even on things that can be determined from Wikipedia, like if there are lions in Senegal (yes) or elephants (nope). AI may get better this eventually, but I'm surprised that it does poorly on eg many science topics for which there are accurate resources it could've been trained on.

Answer 17

To answer the title question: I've read an article where someone had basically flipped the question around and gave students a bunch of ChatGPT generated essays to grade, making them aware of the hallucinations etc. it produced. I assume (or at least hope) that as students realize those issues exist, they might become less likely to use ChatGPT.

Answer 18

The use of AI by everyone (including students) is inevitable because it increases the productivity of work. Actually, students now study the ways to use AI effectively instead of study the class subjects and training the skills of thinking. This is the problem, not the use of AI in classes itself. I don't see how to fix this problem without principal changes in the process of teaching and grading.

Answer 19

Stop grading homework

I didn't study languages at university, but computer science. We had this issue much earlier, since Google and Stackexchange could help you cheat a lot already 10-15 years ago. The way they solved this issue in our university was to not grade homework.

They would grade how often homework was submitted, but not the content of the homework.

Instead, a lot of courses transitioned from a lecture-style setup (frontal lectures with homework) to a exercise-style setup (in-classroom exercises that get graded) or at least a hybrid setup where both styles are used in a course.

Stop grading things AI does well (at least in homework)

This is a bit more tricky. Remember roughly 20 years ago when writing homework on PCs got popular and everyone was worried that spellcheckers helped students to write better stuff than what they could without tools?

In the end, the response was to move the base level. 30 years ago, it was even OK to have spelling mistakes in professional communication. Nowadays you are expected to run a spell checker over a social media post.

Correct spelling isn't really a skill anymore, but a requirement. And it makes no sense to tell students not to use a spell checker, because in every written communication in their real life they will use a spell checker.

Similar things happened in maths with the introduction of the calculator.