17

The situation I have in mind is concerned with teaching introductory probability and statistics to business students, and there may be similar situations in other fields. Because of the risks associated to cheating in exams where computers are allowed, exams are restricted to basic hand-held calculators and datasets are consequently very small : compute mean and standard deviation for less than 10 data points, etc. As the material gets more complex (regression analysis) the sole option is to ask about analysis of printed computer output.

As a consequence, a lot of time is spent both in class and in student study time on learning such useless skills as computing standard deviations with a calculator (even I get it wrong half the time). This in turn leaves less time to teach what actually is useful : doing basic analysis with the computer (Excel, to start with). Another consequence is that it turns the students off any interest for the material as they understand that what is taught in class has little relevance for practice.

In short:

  • time is wasted on outdated methods
  • useful skills remain untested and mostly unlearned
  • relevance is lost (course is "boring")

I would rather avoid complex, time-consuming and costly methods of computer surveillance. Blocking wifi (jamming) internet access during exams is not an option due to local legislation and there is no access to a lab (students have to have their own laptops). Having the students use their computers as cheat-sheets is less of a concern.

So my question is: how can I improve the situation?

The main objective is to bring in more computer-assisted skills while limiting the opportunities for cheating.

  • 1
    Computing standard deviation by hand is something that can be taught in a classroom. Computing standard deviation in Excel or Matlab is something that can be taught in a lab (if this course has lab-time). I am not sure you can do both. – Alexandros Feb 19 '15 at 4:34
23

My undergraduate multivariable statistics class was designed like this: there were 5 larger computer labs, where we got to analyse large amounts of data using various computer programmes to try to answer some questions. We found out in advance that there were 15 sets of data, and 5 versions of the questions, so everyone had a different assignment. These labs were used to check our computer skills.

During the final exam, we were tested on theory: proofs, deriving formulas, explaining which statistical test we would use for some hypothetical situation, and one question on interpreting the output from the statistics programme (MINITAB) that we used the most. This way, we were tested on the computer skills in a setting that was very similar to how we might use them in our working life, and the exam was pure theory: I never even opened my calculator during the final exam. Giving us different data sets, as well as requiring a full lab report analysing the output ensured that we actually understood what was going on with the computer and made it very difficult to cheat.

  • 5
    +1 I think this approach works very well. We also had a pure theory final but in our case the homework consisted not only of theory and by hand calculations but also of various problems that where to be solved with a computer (excel/libre office, and R-programming) and you had to earn a certain amount of homework points to be admitted to the final, so everybody was forced to do at least some of the computer stuff. We did have a TA who gave a 1h introduction whenever we needed a new piece of software or programing language. – Sumyrda Feb 19 '15 at 7:19
15

A common approach (not useful in your specific case, but I'll leave it in case it is useful to others) is to hold the exam in a university computer lab, where you control the hardware and software on all the computers. With the cooperation of your IT staff, you can install all the software they need, and then do some of the following:

  • disconnect the computers from the Internet (by disabling the connection in software, or physically pulling plugs)

  • install selecting Internet-blocking software

  • install keyloggers or similar tools, whose logs you can use as evidence if there is any misbehavior

  • walk around and watch what students are doing - so that they aren't opening administrative tools, rebooting from USB drives, searching the Internet with their phones, etc.

(I presume the "local legislation" says that you can't use RF jamming to prevent students from getting wireless Internet access on their own devices. But surely you can block Internet access from university-owned devices; there can't possibly be a law against that.)

You've added the information that your university doesn't have computer labs; students have their own laptops. I believe there is such a thing as blocking software that can restrict access to the Internet, other applications, etc. You could require that students temporarily install such software on their laptops. This is kind of obnoxious, especially for students using alternate operating systems or other unusual setups, and it won't stop a determined student (who could, for instance, set up their laptop so that the software appears to install but doesn't actually block anything), but it may be worth considering.

  • 9
    This is good advice, but talk to your IT staff early. If they are good, they can tell you how to do this much better than you would, and point out all sorts of problems you might not see. (For example, I was surprised to learn here at Michigan that we couldn't run MATLAB if we killed the network connection, because it checks with the license server too often.) – David E Speyer Feb 19 '15 at 2:28
  • 1
    @Nate Unfortunately there are no more labs in my school, students have to have their own laptops. I will add that to the question. – A.G. Feb 19 '15 at 4:17
  • 1
    @A.G.: Ok, too bad. I added another idea though it's not as good. – Nate Eldredge Feb 19 '15 at 6:20
  • 3
    An alternative to blocking software (which is rather obnoxious) is remote access software, with which students could temporarily give you access to their screens so that you can check on what they're doing. There are a variety of cross-platform solutions that don't require a full blown installation (i.e. you just download something and run it without installing) – ff524 Feb 19 '15 at 6:38
  • 2
    @NateEldredge - Even if that is true it is the nature of the type of software you are talking about that makes it very vulnerable. And it passes its vulnerabilities to the students laptop. Basically you are telling the students to load spyware. I was in the security field and I would never accept any software like this being loaded. I mean seriously the students load it, it infects their machines, it infects university machines, and then as the professor... you just say ooopps. – blankip Feb 19 '15 at 15:07
9

You could actually do this with the student's own laptops if you are willing to do the following:

  • Have the students use < name_of_statistical_software > in Linux on the exam. Assumes that this software runs on Linux. In this case you should have them use it in Linux also during the semester, so they are familiar with it.
  • Buy one 8GB memory stick for each student (plus a few spares).
  • Have someone in your IT department roll a Ubuntu (or whatever you like) live USB image that has all networking support removed.
  • Hand out these USB sticks a few minutes before the start of the exam.
  • Make sure that everyone's laptops can run said live USB image, and provide a few (probably 5-10 depending on class size) university laptops for those students who have laptops incompatible with your live USB image.

Bonus: not only can you deny them internet access, you can also deny them access to the hard drive of their computers during the exam, if the live USB image is setup to require a root password (which they don't have) for mounting hard drives.

Second bonus: this way their computer will be back to normal as soon as they reboot the computer and remove the USB stick.

7

You could try a written exam with questions like this:

Here is a description of an experiment. The experimenters gathered 10000 data points and analysed them using R. Here is a printout of the R session.

  • Describe in words the tests that were carried out.
  • Which of these tests were appropriate to this situation, and which were misguided?
  • The experimenters carried out a foobar test with these parameters, and then changed their mind and used those parameters instead. Which was correct?
  • Can we conclude XYZ? Does it make a difference whether XYZ was specified as a hypothesis before the experiment?
5

Not having a university computer lab is absolutely painful. However, you can improve the situation:

  • Make computer-assisted assignments for the relevant section. Ideally you would have an original, challenging assignment which is difficult to complete even with the internet available, and where Googling the answer is not of use.
  • If conducting an exam, put a perceptive tech-familiar exam supervisor in the same room to make sure the students are not cheating.
  • Ask students to turn off mobile phones and other electronic devices in the exam room
  • In the exam, assess computer skills in a written form: "What would you do in this situation?" "How would you solve X with Y program?" "What does A function do in B program?"
  • Allow the students internet access! You mention 'skills that would be useful in practice'. In practice, you do have internet access, and it's useful to learn to use it well! (Keep in mind you will have to monitor them to stop them communicating with other people.)
  • 3
    I’ve sat an exam from C programming for UNIX where we were allowed to use our own laptops, connect to the Internet and use anything we found. Only communication with any other people was forbidden. Those who did not know what they were doing gave up the exam anyway. The exam was challenging from the point of task decomposition, program design and POSIX API principles knowledge. I used manpages extensively to not have to remember names of all functions and order of parameters, and I googled solutions to several really minor problems. I really liked the approach of the examiner. – Palec Feb 19 '15 at 21:43
3

I don't know if they could be set up for suitable questions, but some automatic assessment programs can be set up for 'proctored exams', where the students can't leave the browser while still taking the test. I've not used this myself, but I've seen the options available.

3

Maybe the right approach here is to have a smaller exam mainly on the rest of the material (is there any?) and also a big home project / assignment.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.