Can anyone recommend an open source system for students to submit programming assignments? Ideally it would be something that would run on the web, do some level of testing on the server side, and provide feedback to the students based on testing.
- Create a repository for the homework where students can submit pull requests to.
- Make scripts to run and test the solution and put them on the repo.
- Students would then fork the repo, add their submission, and make a pull request.
- Setup Travis CI to run the test scripts on the solutions. This will require some form of standardization for the commands to compile and run the solutions.
The nice thing about this system is that the pull request page on GitHub reports the status of the Travis CI run. You can then leave comments for the students and they can iterate until Travis gives them the green light.
It has been used also in the international informatics Olympiad and several other international programming contests, so it is mature and well tested.
It is in use since a few years in the undergraduate course on Algorithms in our CS degree. It allows students to submit their source code online, where it is executed in a sandboxed environment and scored automatically on the basis of some test cases. It supports out of the box C, C++, Java, Python, Pascal, PHP, and other languages can be added (although I don't know how easy it is).
Github classroom provides a free place to make and submit assignments and you can set up autograding. One thing is that in github with github actions you can also build tests right into the push process. However, I have seen some people on the forums for teachers say that they run out of free processing time during busy parts of the semester.
I can strongly recommend PASS (Programming Assignment Submission System), which was written by Mrs Marsupial to support my teaching on first and second year programming modules (Java and C++). It formats the code into a .pdf document using LaTeX, but it also compiles and executes the program and adds the compiler messages and output to the .pdf file (and any output files that the program is supposed to produce). The .pdf file includes an embedded .zip file containing the code in a standardised format in case further inspection/testing is required. The requirements for each assessment are provided in an XML file for each module that specified expected files, input datafiles etc. We have done some experimentation with automated testing, the simplest approach would be to include a Makefile and test harness in the metadata and PASS can add the output of the test harness. However, for the modules I teach I am mostly interested in the structural side of the program design and whether the program is maintainable, so I tend just to have output specifications that show the program basically works and then mark by annotating the .pdf file on my ipad.
We have been using PASS for quite a while at my institution and it has saved a lot of unnecessary admin and hassle. The major benefit is that there is a well-defined target environment (eliminating the "well it works on my computer" complaint) and the students get to see exactly the output of the program as I see it when it is marked. Unfortunately students have to formally submit the .pdf file via BlackBoard at my institution, but another useful feature of PASS is that it provides a reliable independent time-stamp that can be used to determine whether an academic advantage has been gained by a late/failed BlackBoard submission.