9

It blew my mind when I found students can use multiple AI tools to overcome the possibility of detecting the use of AI to answer assignments and test questions.

I was on a family visit and sitting with several guests. One of the people I distantly knew was doing their assignment. That personally was literally copying the question from the assignment and pasting it to ChatGPT to find an answer. After receiving an answer from chatGPT, the person again copied the answer from chatGPT and pasted it to the premium version of another AI tool to expand or shrink the length of the answer and change wording, tone, and such. After using 2-3 AI-based tools to write an answer, the person then uses another AI-based tool to plagiarism checking software to check plagiarism. After ensuring there is no plagiarism as suggested by AI tool, the person submitted an answer with scientific references. The references cited were actually available online when checked.

This gave me a scary picture of how academia might be operating. For me, it's not that students used AI to answer the question. The major concern is students may not know what they were answering about or what their answer actually means in the real world. What they would do if they had to intellectually think about the same or similar problem in the real world? If students could not solve similar problems in the real world, how that would change the reputation of the instructor and the university?

On the other side, are universities even doing a reasonable job of meeting the market needs and needs of students? If that student who uses AI succeeds in solving similar problems they currently solve using AI in the future, is the university doing a reasonable job of training students in the right direction?

Also, are these academic dishonesties detectable while grading assignments?

I was so confused and speechless seeing this.

6
  • 2
    This is a difficult problem, probably not possible at the moment. It will require a variety of things to find a solution, both technical and regulatory.
    – Buffy
    Nov 15, 2023 at 20:36
  • 7
    And it may wind up requiring new pedagogy as well. Probably not an intractable problem, but unsolved as yet. The CS education community is discussing it.
    – Buffy
    Nov 15, 2023 at 20:41
  • 2
    We may or may not be able to answer this question for now. I certainly think we will be able to in the future. Vote to "Leave open" because it's a real threat and the sub-questions are closely related to the main question. Once there is an answer for the main one, the sub-questions should be relatively easier to answer, hopefully. I am personally eager to know how to detect them because I suspect we already have seen some of them on our site, but they are so hard to be detected.
    – Nobody
    Nov 16, 2023 at 5:01
  • 2
    I'll note that the (early) answers don't deal with detection, but with pedagogical compensation. I think this is all that can be done at the moment. And improvements in pedagogy are always welcome. We don't give assignments to get "answers" but to provoke thinking. Professors can in almost all circumstances provide answers for the questions they ask. But students don't always understand that.
    – Buffy
    Nov 16, 2023 at 14:33
  • AI tool detectors are flawed even with a single use. In any case, we really are in the situation where we are saying "how do I make students not use a calculator when giving them problems". You can't, everyone has a calculator. We need to change the requirements to test knowledge, not ban calculators, its intractable to do so. Nov 16, 2023 at 17:48

5 Answers 5

5

A variation on the Flipped Classroom might help as a compensation since there are no currently viable, reliable detection methods. Like the other answers, so far, this doesn't answer the "detection" question, but is a suggestion to provoke changes in pedagogy to try to keep up with technological challenges.

In a flipped classroom all student (graded) work is done in face to face sessions with the professor and perhaps a few TAs present. Students "study" on their own time with materials provided: books, exercises, videos, whatever. During the meetings they do what was previously called "homework".

To prevent use of AI, you need to use either paper/pencil methods or computers isolated from the net. Both are hard to do. It has been noted that students at the present time aren't really taught how to write by hand, since keyboards have become ubiquitous. Remember Blue Books?

Flipping can be made to scale if TAs are well trained and given the materials to drive sessions. The professor isn't the only one involved.

Several other answers here suggest things like oral exams and discussion meetings. I agree. They can also be important components, provided you can scale them appropriately.

But I think, also, that even with a flipped classroom, more thought needs to be given to this issue.

Finally, I'll again emphasize, we give exercises to students to provoke thinking and learning, not to get answers. We can provide our own answers if we need them. Students don't often understand that. We grad things because students need feedback on their thoughts, not just to categorize them. If AI helps them learn, then I'm all for it. But if it only helps them provide "answers" in a painless way then it is worse than useless. Never mind plagiarism. It is the learning that is important.


A final thought. Imagine a "classroom" in which the "teaching" is done by an AI and the "students" are all just AI bots. Lots of "information" might fly about, but how much "learning" would be done?

3
  • 1
    Re the final thought, AIs teaching AIs is not an absurd thing to imagine and it can result in useful outcomes. For instance, AlphaZero learned to play chess at superhuman level just by playing itself millions of times! Also, smaller models are routinely being trained on the output of bigger, more powerful models in order to cut down inference costs.
    – Polytropos
    Nov 16, 2023 at 21:08
  • @Polytropos, you miss the point entirely. It may be a way to advance AI, but it isn't how students learn anything. If I sit in the dorm and send my AI to class, what have I gained? How has my mind improved?
    – Buffy
    Nov 17, 2023 at 13:56
  • It depends what you then do with the AI. If the AI can collect information on a topic (maybe by talking to humans, reading things it finds on the web, talking to other AIs, talking to itself) and then present it to the user in a form that is useful to them, this could be great for the user's mental improvement. GPT-4 can do this to a small degree. From more specialised AI, humans can often learn a lot already (for instance, wide availability of chess computers has without a doubt raised the level at which players with a serious interest in the game play chess).
    – Polytropos
    Nov 18, 2023 at 10:45
7

I've heard of educators who are requiring writing be done in Google Docs and shared with them. They then check the revision history to ensure that each student actually typed the words and made revisions instead of pasting it all at once. It's probably still possible to cheat this system, but not without both planning and effort. Some will try. After all, there are plenty of memes out there about students who've copied a ChatGPT essay by hand onto paper (but forgot to remove the part where they state they're a large language model).

See Using Google Docs Version History for Good (and Evil).

3
  • Using Google Docs will not be allowed by current policies of many Universities in the UK (with obscure references to GDPR etc). Even when it is blatantly obvious that student plagiarised, University due process will often make it impossible or too costly for the academics to effectively pursue the case. Nov 16, 2023 at 14:08
  • 1
    @DmitrySavostyanov In the US, it's common to use Google products even in K12 — the school email will even be gmail. Otherwise, Microsoft products are used, and I think revision history works the same in Word. I can't speak for the rest of the world, do you really not use any cloud-based suite of tools?
    – Laurel
    Nov 16, 2023 at 15:16
  • @Laurel Cloud services are used a lot, but Universities typically pay for the service, keep the data in house (or in the UK), and force the service on all staff. For example, if Uni admins decide that assignments are submitted via Blackboard and TurnitIn, academics can't require students to share Google Doc with them Nov 16, 2023 at 19:00
6
+50

This is something many, if not all, departments are thinking about. As of now, we cannot detect the (smart) use of AI. What I have seen is that many are rethinking the way they do their assignments.

My department is thinking again about what we want to teach our students and what not. This includes how to effectively use AI, and learn when they can and cannot rely on it. It takes time before that really permeates in all courses, and not all first attempts will be successful (and probably many second attempts will go wrong too). We have to learn how to handle this new situation.

If we want to test if students can do something or know something without the aid of AI, then we can no longer let them do it unsupervised. No more take-home exams, no more essay that can be written at home, etc. That is a pity, but that is the state of the world.

If the goal of the assignment is to give them experience and practice without AI, then we just have to communicate that with the students. They are adults and responsible for their own education. In those cases, the teacher gives feedback but no grades. If the student still uses AI in that case, they only hurt themselves. (Not quite, it also wastes the time I use to give feedback on AI generated stuff, but it is not a perfect world.)

If the goal of the assignment is to give them experience how to use AI, then there is obviously no problem with students using AI.

1
  • I agree to everything in this answer. Unfortunately, especially post-covid, my University has strongly encouraged us to move away from in-person exams as they cause students stress and anxiety.
    – penelope
    Jan 11 at 13:54
4

Make the essay compulsory but do not mark them (a simply adequate/inadequate mark), then decide your marking on an oral exam.

It works for very specific exams like n-dimensions manifold as well as for the first Algebra course. It is more work for the teacher, but there are solid reasons to have a teacher doing the teacher job and what you mention is a reputation problem for the university, the same university has to prevent it.

Too many students for a professor to have an oral exam with each one of them? This is again an university problem, it is time to hire more people and less manager to take care of their "clients".

9
  • "Make the essay compulsory but do not mark them (a simply adequate/inadequate mark)…" I don't understand; are you marking/grading the essay or not? And what is the motivation for a student to write a good essay?
    – Laurel
    Nov 16, 2023 at 14:36
  • Yes, oral exams don't scale well, and TAs may not do a good job in giving them. But an oral exam need not last for more than one or two questions and a few minutes. After all, the normal grading schemes normally reduce to only broad distinctions - letter grades.
    – Buffy
    Nov 16, 2023 at 14:37
  • @Laurel, the motivation, at least for the better students, is to learn something, not to provide an "answer". Again, we ask students to do exercises, not to get answers but to provoke thinking. But it can be difficult to teach that particular lesson to some students. Ideally the motivation is internal, not external.
    – Buffy
    Nov 16, 2023 at 14:40
  • 1
    "this is again an university problem" ...sadly, many universities will not (or cannot) put more money into the matter to solve it, which makes it your problem again..
    – user111388
    Jan 11 at 11:35
  • 1
    @TheDoctor an inadequate mark is "inadequate to be admitted to the exam". A blank is surely inadequate, a half-thought essay may be barely adequate, or extremely inadequate, an AI essay is surely adequate and it must be backed by equally insightful sentencing and knowledge at the exam. In short: if you do a bad, but adequate, essay you may get out of the exam with a bad, but adequate mark. If you do an excellent essay, then at the exam you must confirm the excellence. Otherwise I fail you.
    – EarlGrey
    Jan 12 at 7:24
3

The major concern is students may not know what they were answering about or what their answer actually means in the real world.

One method to find out whether that's the case is to have a discussion meeting for the assignment, in which the student needs to answer critical questions about their answers. At a minimum, the student must be able to exactly explain what they wrote, without looking at their answer. One could also require them to answer at least a few questions that are a bit to the right and left from the path of their argument.

Of course, the drawback with this is that it requires resources and might not scale nicely to large courses. One mitigation could involve TAs and having some "pre-filtering" through discussions between TAs and students, where only in critical cases students need to discuss with the main teacher.

3
  • Another good idea, but for the scale factor.
    – Buffy
    Nov 16, 2023 at 14:41
  • This could actually scale if the individual discussions about the essays were done by an AI that has been briefed about the subject by the instructor, especially on parts of the topic that the AI would get wrong on its own (so the examiner-AI would have a knowledge advantage over any AIs that students might access). The instructor would still have to review chat transcripts for which the AI suggests a grade change, but human workload might be quite manageable in such a system, although I can see this idea is probably an utter nightmare for people who dislike AI.
    – Polytropos
    Nov 16, 2023 at 21:05
  • One thing I have done successfully is ask about five randomly selected students to explain their code to me. I say that in my syllabus and tell the students that they are randomly selected and selection does not mean that I suspect that they have cheated. This keeps the workload balanced.
    – unxnut
    Dec 11, 2023 at 2:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .