How can I balance teaching computer-assisted skills and risks of cheating at exams?

Question

The situation I have in mind is concerned with teaching introductory probability and statistics to business students, and there may be similar situations in other fields. Because of the risks associated to cheating in exams where computers are allowed, exams are restricted to basic hand-held calculators and datasets are consequently very small : compute mean and standard deviation for less than 10 data points, etc. As the material gets more complex (regression analysis) the sole option is to ask about analysis of printed computer output.

As a consequence, a lot of time is spent both in class and in student study time on learning such useless skills as computing standard deviations with a calculator (even I get it wrong half the time). This in turn leaves less time to teach what actually is useful : doing basic analysis with the computer (Excel, to start with). Another consequence is that it turns the students off any interest for the material as they understand that what is taught in class has little relevance for practice.

In short:

• time is wasted on outdated methods
• useful skills remain untested and mostly unlearned
• relevance is lost (course is "boring")

I would rather avoid complex, time-consuming and costly methods of computer surveillance. Blocking wifi (jamming) internet access during exams is not an option due to local legislation and there is no access to a lab (students have to have their own laptops). Having the students use their computers as cheat-sheets is less of a concern.

So my question is: how can I improve the situation?

The main objective is to bring in more computer-assisted skills while limiting the opportunities for cheating.

Answer 1

My undergraduate multivariable statistics class was designed like this: there were 5 larger computer labs, where we got to analyse large amounts of data using various computer programmes to try to answer some questions. We found out in advance that there were 15 sets of data, and 5 versions of the questions, so everyone had a different assignment. These labs were used to check our computer skills.

During the final exam, we were tested on theory: proofs, deriving formulas, explaining which statistical test we would use for some hypothetical situation, and one question on interpreting the output from the statistics programme (MINITAB) that we used the most. This way, we were tested on the computer skills in a setting that was very similar to how we might use them in our working life, and the exam was pure theory: I never even opened my calculator during the final exam. Giving us different data sets, as well as requiring a full lab report analysing the output ensured that we actually understood what was going on with the computer and made it very difficult to cheat.

Answer 2

A common approach (not useful in your specific case, but I'll leave it in case it is useful to others) is to hold the exam in a university computer lab, where you control the hardware and software on all the computers. With the cooperation of your IT staff, you can install all the software they need, and then do some of the following:

• disconnect the computers from the Internet (by disabling the connection in software, or physically pulling plugs)

• install selecting Internet-blocking software

• install keyloggers or similar tools, whose logs you can use as evidence if there is any misbehavior

• walk around and watch what students are doing - so that they aren't opening administrative tools, rebooting from USB drives, searching the Internet with their phones, etc.

(I presume the "local legislation" says that you can't use RF jamming to prevent students from getting wireless Internet access on their own devices. But surely you can block Internet access from university-owned devices; there can't possibly be a law against that.)

You've added the information that your university doesn't have computer labs; students have their own laptops. I believe there is such a thing as blocking software that can restrict access to the Internet, other applications, etc. You could require that students temporarily install such software on their laptops. This is kind of obnoxious, especially for students using alternate operating systems or other unusual setups, and it won't stop a determined student (who could, for instance, set up their laptop so that the software appears to install but doesn't actually block anything), but it may be worth considering.

Answer 3

You could actually do this with the student's own laptops if you are willing to do the following:

• Have the students use < name_of_statistical_software > in Linux on the exam. Assumes that this software runs on Linux. In this case you should have them use it in Linux also during the semester, so they are familiar with it.
• Buy one 8GB memory stick for each student (plus a few spares).
• Have someone in your IT department roll a Ubuntu (or whatever you like) live USB image that has all networking support removed.
• Hand out these USB sticks a few minutes before the start of the exam.
• Make sure that everyone's laptops can run said live USB image, and provide a few (probably 5-10 depending on class size) university laptops for those students who have laptops incompatible with your live USB image.

Bonus: not only can you deny them internet access, you can also deny them access to the hard drive of their computers during the exam, if the live USB image is setup to require a root password (which they don't have) for mounting hard drives.

Second bonus: this way their computer will be back to normal as soon as they reboot the computer and remove the USB stick.

Answer 4

You could try a written exam with questions like this:

Here is a description of an experiment. The experimenters gathered 10000 data points and analysed them using R. Here is a printout of the R session.

• Describe in words the tests that were carried out.
• Which of these tests were appropriate to this situation, and which were misguided?
• The experimenters carried out a foobar test with these parameters, and then changed their mind and used those parameters instead. Which was correct?
• Can we conclude XYZ? Does it make a difference whether XYZ was specified as a hypothesis before the experiment?

Answer 5

Not having a university computer lab is absolutely painful. However, you can improve the situation:

• Make computer-assisted assignments for the relevant section. Ideally you would have an original, challenging assignment which is difficult to complete even with the internet available, and where Googling the answer is not of use.
• If conducting an exam, put a perceptive tech-familiar exam supervisor in the same room to make sure the students are not cheating.
• Ask students to turn off mobile phones and other electronic devices in the exam room
• In the exam, assess computer skills in a written form: "What would you do in this situation?" "How would you solve X with Y program?" "What does A function do in B program?"
• Allow the students internet access! You mention 'skills that would be useful in practice'. In practice, you do have internet access, and it's useful to learn to use it well! (Keep in mind you will have to monitor them to stop them communicating with other people.)

Answer 6

I don't know if they could be set up for suitable questions, but some automatic assessment programs can be set up for 'proctored exams', where the students can't leave the browser while still taking the test. I've not used this myself, but I've seen the options available.

Answer 7

Maybe the right approach here is to have a smaller exam mainly on the rest of the material (is there any?) and also a big home project / assignment.