6

I will be teaching a course that has a significant programming component. I would like to evaluate students' programming skills using a computer-based quiz/exam, because it seems to me that this is a better way to evaluate students' programming skills than a paper-based quiz/exam, given that in the real world, people write code on a computer, not on paper.

Unfortunately, I have never attended a quiz/exam where programming was done on a computer, and the computer was connected to the Internet. (I think I did take a computer-based SAT or GRE exam many years ago, but I don't think the computer was connected to the Internet.) I am worried that in a computer-based setting, students may be tempted to cheat using e-mail or Internet messaging software. Is there a reasonable setup that I could use that allows students to work on the computer, and submit their work online, while limiting the ability of the students to use Internet communication?

  • 4
  • 2
    I would question your assumption about using a computer. Who cares whether taking the test resembles the "real" world? It's a small fraction of students' time, spent proving their comprehension of specific topics. My own solution for CS exams is to give conceptual questions the answers to which are difficult to look up, and accept answers either on paper or in email. – Aaron Brick Aug 25 '16 at 2:57
  • 2
    Why would you even need internet connection? You can access the documentation of common programming languages from any modern IDE. And if we assume that the given programming tasks don't require anything (algorithms, design patterns, libraries, etc.) new, i.e, which hasn't been shown before in the course, then the students shouldn't have to resort to googling. Plus, a good IDE can be set to have debugging and performance analysis capabilities, and maybe even also the ability to generate various graphs/reports from the code. And if you think this may be too daunting for beginners, you can... – 101010111100 Aug 25 '16 at 6:23
  • ...always seed the IDE to include things like project configuration, libraries, empty source files, etc beforehand. Or you can bring it a step further by providing mostly completed source files and require that students simply fill in any missing pieces of code (e.g. a particular algorithm), or even provide fully completed source files and require that students refactor them to make use of certain design patterns, or debug/profile your code and find a bug/inefficiency. The possibilities are endless ;). – 101010111100 Aug 25 '16 at 6:31
  • 1
    It's low tech, but is there a chance to do the exam in a computer pool in which you can see the displays? E.g., if the computers are arranged in a typical classroom setting, to stand on one side of the room, looking at the screens? Or (available in some computer pools) to use a monitoring software that lets you see the students screens (without them knowing which one you focus on)? – Daniel Wessel Aug 25 '16 at 10:46
4

If you have control over the lab software

Have your IT dept install software that will allow you to monitor the activity on each system. I think the term is Classroom management software. Sit back and watch for suspicious activity.

Otherwise - adapt

I like the strategies discussed in this question about take-home exams (How to prevent cheating on take-home exams)

The TL/DR - think about how you might test learning of the material in a situation where students ARE going to collaborate and ARE going to access materials on the subject.

My two cents on the concept of computer based exams

it seems to me that this is a better way to evaluate students' programming skills than a paper-based quiz/exam, given that in the real world, people write code on a computer, not on paper.

I don't agree with this statement, because in the real world people also have access to other supports like references and peers while writing code on a computer.

I don't think writing a coding exam on a computer actually tests knowledge of coding any better than writing code on paper. In fact, in some ways it hampers student performance because they must focus on creating properly formatted, compilable, code instead of getting their knowledge of the concepts on paper. Pseudo code is faster to write and easier to evaluate.

If you want to test their ability to write executable code, give them assignments.

|improve this answer|||||
4

it seems to me that this is a better way to evaluate students' programming skills than a paper-based quiz/exam, given that in the real world, people write code on a computer, not on paper.

If you want your exams to better approximate the real world, then speaking as an IT professional, let me make a few comments.

The Real World

  • In the real world programmers have access to documentation.

Not only do they have access to documentation, they are expected to use it and refer to it regularly, if not constantly. It is a red flag to any good technical manager to have a programmer waste time trying to "figure something out" by trial and error rather than consulting the documentation or other online information.

(Of course, it is also a red flag if the programmer can't write a simple "for" loop without consulting the documentation.)

Yes, it is an advantage to know your language so well that you can use all the standard library features without consulting any references or examples. But few programmers are that good—and the ones who are, got that way by practice (experience), not by reading textbooks or doing classroom assignments.

  • Writing the actual code is a very small part of designing a solution.

Design questions are important in the real world. Vast books have been written on the subject. It's not enough that your code works. It must be sufficiently robust for the conditions it will actually run in; it must be maintainable; it must be possible to modify it to accommodate new requirements without throwing the entire codebase away and starting from scratch.

Design also includes understanding the requirements and can even include querying them (proposing alterations) or surveying the source of the requirements to be sure they are really, truly understood before a solution is implemented. A classroom setting almost never simulates this accurately. In class, you are given a precise assignment, and you must solve exactly the use case precisely (even rigorously) stated. The real world isn't like that.


A standardized exam to be taken during class should probably not focus on practical programming. Take-home extended assignments would be a much better gauge of ability to write code, as abase states in his answer.

|improve this answer|||||
2

A feasible solution would be to use firewalls that can be configured to allow only certain sites, such as the ones required for the exam, to connect over the Internet and block all else. This would be able to thwart Internet based messaging and web referencing.

If you are concerned about the possible communication over the intranet, for instance a telnet connection which is to be part of the programming examination, there are software which could control code plagiarism. For example, Moss from Standford is an automated system that determines the similarity of programs. I've also heard of some that would be able to check the similarity at the opcode level in such a way that even renaming variables wouldn't allow perpetrators to escape.

|improve this answer|||||
  • -1 for the first paragraph (focuses on the wrong problem), +1 for the second. – Federico Poloni Aug 25 '16 at 8:30
  • @FedericoPoloni: Thank you for the comment. The first paragraph was intended to address the point "...tempted to cheat using e-mail or Internet messaging software." to thwart copying if it occurs. Do you still feel it is irrelevant? Or should I make an edit to make this clear? – Ébe Isaac Aug 25 '16 at 8:36
  • 1
    Sorry to be blunt, but yes, I still feel it is irrelevant. Plagiarism detection software does not catch a friend helping me using Gmail chat. – Federico Poloni Aug 25 '16 at 8:38
  • @FedericoPoloni: agreed, I've made the necessary edit. – Ébe Isaac Aug 25 '16 at 8:49
1

Ask your students to copy an honor statement, e.g., "On my honor, I will neither accept nor give unauthorized aid", then add their name.

I report students for academic misconduct every quarter but I have never had to do it on a test where I've asked them to copy an honor statement. When people are reminded of their ethical obligations, they become more honest, consistent with what Dan Ariely at Duke reports in the (excellent) documentary, "(Dis)Honesty: The Truth About Lies".

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.