Creating a secure test environment for a programming lab practical

Question

I'm a senior TA helping out my instructor with restructuring the course in light of the coronavirus causing in-class cancellations at our university. We have tools allowing our students to do their programming labs online, and therefore our class can still run, but we still have a lab practical that's a huge part of students' grades, and I personally would prefer that that not be replaced by some generic homework.

Would any of you happen to know of ways to make secure environments where a website (in this case, the tool allowing students to do their labs) could be loaded? The student shouldn't be able to navigate to another tab or close full screen, and the website immediately submits code to the server when the time for the virtual "exam" is up.

I've tried searching for examples myself, but I'm not sure if I'm wording my problem correctly because I don't get any relevant results. I'd really appreciate the examples you have. Thank you in advance!

Answer 1

As already outlined in some of the comments, it is impossible to achieve what you want: You cannot control a computer to this extent without a severe invasion that would go far beyond anything that I would allow you to do on my computer due to privacy and security concerns. Unless you want to video-monitor students, you can also not prevent students from using a second computer, a virtual machine, or having somebody else working on their computer for them. And even then you need at least two cameras to see everything that the student is doing or that is happening to them and you would have to frequently check that they are working themselves, did not rig the video, etc.

On a more general level, controlling cheating on any exam that is based on limiting resources is already difficult if you do it in a setting that you can control (i.e., your room, your computers, your toilets, etc.). By taking this battle to the “enemy’s territory” and limiting your knowledge, you can only lose.

I would thus suggest the following alternative. This is how my programming practicals were done when I was a student. I also successfully applied this to a programming “exam” in a course I co-organised.

1. The students do the programming practical with no restrictions except time. For example, they may use the entire Internet, course material, etc.

2. After they submit the code, there is a short oral exam on the code. Depending on the details of your course, you can very this exam in length and relevance, with the extreme examples being:

   • You grade the code and the oral exam only serves to ensure that the students didn’t cheat (and may be as short as five minutes).
   • You only check that the code is doing what it’s supposed to do, while the oral exam determines the grade and may also contain questions about other aspects of the course and deeper understanding. Here, the oral exams may cost more time, but you also spend less time looking at their submissions.

3. If applicable, also perform plagiarism checks on the code.

Some pros and cons of this approach:

• You somehow have to organise remote oral exams, but these could be just via the phone, if that suffices to establish the examinee’s identity for you. Also be prepared that this mode will amplify anxiety, language, and other issues for some students (while it reduces them for others).

• You may need to modify your tasks to account for full Internet access and similar, for example it should not be anything to which a bazillion of solutions exists. Depending on your topic, this may be very difficult.

• The exam is more realistic: except for taking exams, your students will have Internet access for the rest of their career and using it properly is also a relevant skill.

• The oral exam allows you to gauge actual understanding, whereas the purely written exam may be solved by rote-memorising algorithms, etc.

• In case of group work, you do not need to worry much about some incompetent student benefiting from a stronger partner. (I am aware that this point doesn’t apply to you; I mention it only for completeness.)

Answer 2

You are looking for a remote proctoring service, of which there are many

Unfortunately, these are usually enterprise level services which your university would have to contract for.

Background

I also use in class programming quizzes which I think are equivalent to what you mean by practicals. And I'm also trying to adapt to the new reality with a course that was never intended as an online course, and would have been set up differently if it had been.

The assignments are short, simple programming concepts like constructing a basic loop within a fixed period of time. In some years I have had the students email me their completed script, but for the past few years I have had them upload their script to the course LMS (my school happens to use Canvas). In the LMS, the assignment is constructed as a quiz. They open the quiz, see a question which is created as a "File Upload Question" (i.e., the "answer" is a file uploaded by the student), copy starter code to their programming environment, complete the script, save it, and then upload their answer.

Remote Proctoring

My university has provided us access to a remote proctoring service that can do what you are looking for. Note that I am not endorsing any particular commercial service, nor did I have a hand in my university's selection. Therefore, I am not naming the service we use. A variety of such services exist, and can be easily discovered through a search engine with "online remote proctoring" or something similar as your search term.

These services usually integrate with the major LMSes (Blackboard, Canvas, etc.) and may offer the following features, which can be configured for each quiz/assignment:

• Video recording
• Audio recording
• Screen recording
• Logging all web traffic
• Forcing all browser tabs other than the quiz tab to close, and preventing opening other tabs (note, this is easily defeated if the student has a second browser installed, so I don't know how useful this is if you don't also have screen recording or web traffic logging turned on)
• Identity verification (such as through an school ID card, video captured)
• Live proctoring provided by the service

Auto-submission when the time limit is reached (your other request) is built into the LMS. Note, however, that the students are not coding in a web IDE, so in my setup an auto-submitted assignment is just blank, because it means the student didn't attach their completed script file in time.

The problem with such a solution is that most (all?) of these are enterprise level solutions which an institution subscribes to rather than an individual. So if you don't already know whether your university has a subscription, you may well not. (But maybe in the flurry of announcements that have been generated this got lost in the mix? Check with your department chair and/or your university's ed tech support.) If you want to use such a service and can't find one that allows individual faculty to subscribe, and your university doesn't have such a service, your only hope (other than coming up with an alternative assignment or an alternative way of evaluating the submissions) is to convince the university to get something going fast. Your voice alone won't accomplish this, but this would be something to mention to the instructor you TA for, or your department chair. If several chairs wanted this, maybe it is something the university could do. On the other hand, enterprise level contracts often take months to hash out and implement, so this may be unrealistic (unless current circumstances make people move faster).

Warning

Some students in my course (roughly 20%) have raised privacy concerns about being videoed while taking the quiz. I'm not 100% certain that this qualifies as a privacy concern, as I warned the students ahead of time and told them to be conscious of their backgrounds prior to starting the quiz, but at least one student has said that it is a continual distraction to be aware of the video and their surroundings. The service we use also has a facial recognition component which a student objected to, although I do not have the identity verification feature turned on (it's a small class and I can see who is taking the quiz). A software which can control your web browser and monitor web traffic can also be characterized as spyware.

If you have any concerns about these features, you shouldn't go this route. If you don't have concerns about these features, your students may nonetheless say "This isn't what I signed up for." If your university doesn't have such a service, it's possible it had been considered already and was rejected or is being held up because of these privacy concerns.

Obviously, if my course was designed as an online course from the beginning, this would all be different, because either:

1. The students would have been made aware from the beginning that a remote proctoring solution was being used, and could choose to take an in person course or a different online course if they had objections to remote proctoring; or
2. I would have designed a different set of quizzes/assignments or a different manner of evaluation where I was less concerned about outside assistance.