31

I am currently teaching a basic R programming course with about 80 students. Under normal circumstances, a core course assessment component is a programming exam, which is done in the computer lab using the lab computers. I will arrange for the Internet to be cut off, and we will have exam invigilators who are patrolling the exam venue to detect or dissuade students from cheating. At the end of the exam, the Internet is restored, and students submit an R script (basically programming code) using the course learning management system.

This semester, due to the novel coronavirus disease (COVID-19), the university where I work has replaced all on-campus teaching activities with online teaching for the entire semester!

  • With regards to teaching, the faculty have been asked to teach our classes online using Zoom, which gives students the freedom to take classes wherever they like.
  • With regards to exams, the faculty have been asked to administer their course exams online, with students completing the exams online from home. (What??!!)

I am very concerned about students taking exams online from home, because I am worried that some or many of the students may use this as an opportunity to cheat on the exam — whether it is collaborating with their classmates, or even asking another student to take the exam in their place.

My questions:

  1. Is there any way to detect if students are cheating (e.g., sharing answers), when students are doing an online quiz or exam at home?
  2. Is there a way to dissuade students from cheating?

Response to comments:

  • How much help do you have for these 80 students?

    I do have one PhD student who is a grader. She is supposed to spend about 6 hours a week to help with the course.

  • 3
    I would have one additional assessment via skype or equivalent and ask a couple of questions direct - you should be able to tell if they are waiting for someone else to help them, then use that grade as a factor for the other grades where they might have had help... – Solar Mike Mar 6 at 14:28
  • theory.stanford.edu/~aiken/moss this might be helpful – sntrenter Mar 6 at 16:33
  • 1
    You include an unusual word in the question. Then, you post an incorrect answer online, including that unusual word. Students google the unusual word. Find the word. Find the answer. Copy the answer. Job done. – Strawberry Mar 8 at 23:51
  • 1
    @Strawberry: I know people who cheated in a similar way. They would not just copy any solution without checking and understanding it. – user111388 Mar 9 at 17:01
  • 2
    I used to give open book, open note tests in my live classes and I found students spent all their time trying to find the answers and didn't finish the quiz/test. Once I switched to closed book their grades went up on the same tests. I'll be randomizing my online tests for each student (x random questions from pool of y, given in a random order), and beyond that I'm not worried. – Kathy Mar 17 at 19:00
65

I also teach a data and programming course, though I mainly use Python.

Is it possible for you to shift grading away from an exam and toward something else? A project, for example? Before going back to academia I spent years working as a policy researcher, and exam conditions simply aren't a realistic test of real-life coding abilities. For that matter, coding with the internet turned off is also not remotely what real work looks like either.

In my class I like to give a larger project and let them have a few weeks to work on it, but I could also imagine a more simple project that they only have a few days, or even hours, to do. Real data work often comes with time constraints, but it does not come with a lack of internet access in exam-like conditions. Just scale the scope of the project based on how long you want them to have to work on it.

There is still a huge advantage to knowing the material, rather than having to look it up. You can also grade on the quality of code and correctness of results, and allow for creativity with open-ended questions like "summarize this data with two plots". This way each student is not only tested their knowledge of R, they're tested on their actual ability to use R in a realistic environment.

If you, unfortunately, are required to give an exam, I know there is software available for proctoring online tests. The sort of thing that utilizes their webcams to make sure they don't leave during the test, or consult other devices, or open other web pages. I've not used them though, so I couldn't make a recommendation. I imagine they all require university subscriptions.

| improve this answer | |
  • 10
    I support the project idea. I think you probably need to abandon traditional exams unless the university makes a serious commitment to a technological solution. That is likely impossible in the time frame required. Some places with a long and strong commitment to honor codes seem to be able to mostly but not entirely control student dishonesty. Dartmouth College comes to mind. – Buffy Mar 6 at 15:32
  • I would prefer paired or small group projects myself. Collaboration is probably hard to prevent, so if you make it required you don't have a problem. – Buffy Mar 6 at 15:33
  • @Buffy In general, yes. But the OP said that because of the COVID-19 virus, the university is closed (presumably to prevent unnecessary contact between persons). So in this case forcing cooperation would not fit well... – Maarten Buis Mar 6 at 16:03
  • 3
    This would be a fine idea if designed into the course from the outset, and announced in the syllabus. Unfortunately in this case the course is already underway, and most likely the syllabus says that the grade will include an exam, not a project. The syllabus is often treated as a "contract" with the student, and any significant deviations may result in well-justified student complaints. – Nate Eldredge Mar 8 at 4:35
  • 9
    @NateEldredge Considering the university just canceled all in-person classes, I think changes to the syllabus to accommodate that seem pretty reasonable at this point. – Jeff Mar 8 at 4:40
16

Make the question non-amenable to Googling

Many of colleagues used to do what you mentioned: Having exams in labs and cutting off internet/network access to isolate their terminals. All with good intentions of not cheating.

A different approach that I implemented was rethinking what I want to test them on and use that as the guiding source and create the problems from scratch. That way, even if they were to Google the question it’s unlikely to find any answer. However, if they are to search for something specific like “Breadth First Search in Java” that is probably okay IMHO. The idea is for them to understand/decipher the problem such that they can correctly infer to use that algorithm. If the implementation language is Java, then they’re free to look up certain implementation details (assuming they’ve had past assignments/practice in some form).

This takes away the focus from the act of “coding the algorithm” to “solving a problem using the algorithm”. The latter is more copy-proof IMHO. Even though they may “copy” the algorithm implementation, they’ll still have to adapt it to the problem at hand and that itself can be a lot of work.

For those open to this “overhead” that’s the choice they make. For those who’re comfortable implementing it from scratch, they won’t rely on this approach. It also takes away the pressure of memorizing/debugging in a very time critical environment and gets them to focus on the act of solving the problem with the intended algorithm.

I’ve had very good success with this approach. It’s more work to come up with these problems and you may run out of them. For that a good strategy is to ask the students to come up with some good problems for the next year and they can earn bonus credit for good submissions - also works like a charm ;)

There are online tools that check for “code similarity” to infer if two people collaborated/shared code. One is MOSS that we used. There are surely others that can help with detecting “cheating/sharing” with great success.

Example:

I have a city map with ‘travel time’ for each bus route between 2 stops. Write a program that asks the user for starting and ending stop and returns the fastest way to get there:

Input/setup:
A -> B: 5
B -> C: 10
C -> D: 1
A -> C: 3

Example IO:
Start-stop?: A
End-stop?: D

Computing...

A -> C -> D: 4

Testing on the ability to find the shortest path - whether they use BFS or Dijkstra is up to them. This is not easy to Google for and even if they do, it’ll take a while to translate to the above problem.

You can timebox the implementation time if you’re using services like Code Signal or the like.

| improve this answer | |
  • 12
    This doesn't solve the problem of students cooperating with other students. – user1482 Mar 7 at 14:23
  • 4
    If one is given an algorithm to code in a specific language, many implementations will differ only in the identifiers. Impossible to distinguish original work from a copy when the plagiarist has only renamed the variables. I once saw an essay where the student had copied a Wikipedia article then modified each sentence enough to hide it from plagiarism-detecting software. (Obviously a very resourceful person, but still plagiarism!) – WGroleau Mar 7 at 15:23
  • 2
    @BenCrowell - agreed. And this is where a timeboxed exam that’ll be held between fixed hours could help. Collaboration by itself isn’t bad IMHO - that’s how folks work in the real world. It’s the act of engineering the solution that should be independent. Tools like MOSS shield against “similar” code rather well IMHO – PhD Mar 7 at 23:42
  • 1
    They weren’t minor modifications, but they were mostly sentence by sentence. Anyway, he was in a high school that did not use any such tools. – WGroleau Mar 8 at 1:26
  • 1
    @PhD my experience of turnitin is rather different. In marginal cases with light paraphrasing it flags sentence fragments that could easily be false positives generated independently (and often from different sources, so there's no block of one colour highlighting with a few white bits), but misses the big picture. Inappropriate synonyms on marking are a better giveaway in that case. – Chris H Mar 9 at 10:02
6

There are tons of open source software or scripts or libraries written in R.

Give your students a randomly chosen such project (not a too small one), with the task of explaining what that software does in written English (perhaps with some math formula in LaTeX or KaTeX), and to extend or customize or execute that software (or library) for a problem of their choice. In other words, ask them for a code review, and if additional time is available, to add one extra functionality or feature (described in written English) to that software.

Such tasks are currently hard to be entirely automated. They might use the Internet, but it won't help them that much....

You could expect cooperation, by giving to each small group of students one additional (but different for every group) feature to add. Notice that such team cooperation is close to industrial practice.

I did teach at University a few years ago. One of the tasks I gave students was to download sash and extend it to add one additional builtin (described in French) in two hours. Access to the internet does not help much in practice. The time was limited, of course.

Today, I am working (in my free time) on RefPerSys. By experience, it is difficult to make motivated persons (which I never met in real life) reviewing some of my git commit-s. I sent them the URL of many papers describing them (e.g. the ObjVLisp model). I am writing emails, submitting papers, draft reports, and slides quite often. We all are very motivated. Even with that, getting code reviews and code git commit-s is difficult. And today, such a task cannot be realistically automatized.

| improve this answer | |
5

I come at this as a former student and as a TA who assisted with exams.

At my college there was one class I took that allowed for students to actively write and submit code via live submission for tests in person. They did not cut off the internet for us, but we were timed, and the requirements for the examination were strict enough that if you did not know the material well, searching for the answers would not get you very good results.

They did this by making sure that you were using a data set that was hand constructed for that test. It happened to have a test in R as well. This data set was unique, and as such, you could not find the answers anywhere online.

As a TA I also know, submissions were automatically compared to each other by the submission system in a way that it could be determined which ones were close matches, so that it would become obvious when 2 people shared answers. These would be flagged for the professor to review.

Altogether, I do not know how effective this system was, as it was fairly new at this point. I was in my senior year, and this was the first time they had tried it, having had paper programming tests only previously.

To summarize, what they tried was to develop an online submission system that uses Closest Strings Matching to determine percentage score matching between student submission, and to develop unique secret problems for each test that cannot be found online in order to have a digital system that can be used in person or online, as well as having a strict but fair time limit to take the test.

As for the success of that system, only time will tell. The school I graduated from is UCF.

| improve this answer | |
3

Disallowing internet searches for a programming course seems a bit outdated these days. I would change the exam so that it relies less on memory and more on technique.

I've had pre-interview technical tests which allowed internet searches, but were hard enough that if you had to constantly look up stuff, you'd never complete it.

Main problem with online exams is that a test taker could just have someone else do the exam for them, even with a webcam pointed at them from their laptop (just have the accomplice use a separate keyboard and monitor).

| improve this answer | |
2

Ask the center for teaching and learning (or whoever at the university is in change of arranging faculty workshops on teaching - all the places I have taught have one) what the options for online proctoring are. When I have taught similar courses, there is software which can be used to proctor online exams. It basically consists of a browser lockdown, recording the screen, and access to the webcam, which records if the student behaves suspiciously while sitting there. Yes, this is pretty intense scrutiny, but it seems to work for our faculty who give exams for online courses.

| improve this answer | |
  • 4
    My oc doesn't come with a webcam, what now? – joachim Mar 7 at 22:32
  • 4
    This is incredibly invasive. First, it almost certainly will require Windows or, at best, MacOS. Secondly, it requires me to allow a very invasive program onto my system. I'm quite shocked that anyone would accept this level of intrusion! – terdon Mar 9 at 12:47
  • 2
    Reiterating @terdon: this seems a very big privacy problem to me. People should never be forced to install proprietary software on their personal computers, unless they had agreed to that previously for that course (and even so, it'd be a very awful policy and the university risks liability if that 3rd party software ends up being compromised). – jjmontes Mar 9 at 19:39
  • The poster asked how other faculty handle things: this is quite common. Even in class exams use browser lockdowns. I would suggest deleting the software after completing the exam, although I suppose if you don’t trust the software you would not trust the deletion process. – Dawn Mar 10 at 0:25
  • Was this a measure just because of Corona or were the students aware when signing up for the university? – user111388 12 hours ago
2

For exams such as AZ-300, Microsoft use a combination of a secured browser and chat using the webcam of the computer being used. The person taking the exam has to scan the webcam around the room to show no other machines are on, and is observed while taking the test. They have dedicated software to ensure that no other browser sessions are open, but most screen sharing software with recording could be used for an invigilator to check that, once the webcam has confirmed there's only the one screen.

| improve this answer | |
  • 3
    This seems a very big privacy problem to me. People should never be forced to install proprietary software on their personal computers, unless they had agreed to that previously for that course (and even so, it'd be a very awful policy and the university risks liability if that 3rd party software ends up being compromised). – jjmontes Mar 9 at 19:38
1

As this applies to the entire university, your case may actually be more amenable to a technological solution than a course where the majority of exam answers are in the lecture notes that the students might have right next to them on paper as they take the exam (so even a secured browser session wouldn't help). The university is likely to be working on a way of handling grades under the circumstances, but it may be too little, too late for you to make adjustments yourself, or it may invalidate those adjustments.

| improve this answer | |
0

From a student's perspective: Don't do time crunched tests that don't really help anyone. Do a project or smaller quizzes. Especially in online classes, tests are the most agonizing task that create SO MUCH anxiety because there is so much that can go wrong that is out of our control. Most students also do better on a honor policy, to trust most students will honorably do their work without the need to spy on them. They will have much less pressure and will learn and retain more. You can design the course in such a way that they are forced to keep up with instruction and that they pay attention to instruction and practice but take the pressure off of them. A little mercy and grace and leniency will mean a lot more than an iron fist and I can say with 100% certainty, your students will retain much more this way.

| improve this answer | |
New contributor
bubbles is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.