homework testing method for schools - how to encourage good practices and design and reduce reliance on proofreading?

Question

I've looked a lot for this kind of question and couldn't find anything, I think the answers here could be helpful to schools.

My collage has problems with it's homework testing system and I'd like to offer them that i'll upgrade or rebuild their testing framework.

My collage testing method is totally black-box. Students write whatever they want as long as they pass the automated tests done on school server.

The teaching assistant then proofreads all students code commenting on faulty logic and bad practices. The problem is that it's not humanly possible to proofread all this code.

My question is this:

What do you think is the best homework testing method that'll encourage (or better, force) students to write good code, and secondly, as a result, will reduce the teaching assistant proofreading task to a minimum.

--We program in C++ on linux.

Thank you for you comments.

Answer 1

You are asking for a fully automated code review pipeline, also known as the Holy Grail.

By parts:

• Faulty logic: this can theoretically be caught by testing. But if it were so easy, software will be bug free. If the assignment is simple enough so that there is one and only one way to do it, you can predict the algorithms, and design corner cases to test for. If there is more than one way (and people, specially beginners will find another way, however convoluted), you would have to review them, see which algorithms they are using, and get mean on them.
• Code style: there are automated tools to check style and good practices, like cppcheck but they are quite stupid. Some settings are over zealous, and they are completely unable to see through the general structure, find too long functions, or enforce single responsibility. Plus, they are designed to give suggestions to the programmer, that he will evaluate if appropiate ("do I really need to document this 3 lines long auxiliar function with a perfectly clear name and signature?").

So, in general, no, that is not possible. The only way I can think is to make it easy to expand your testing and evaluation framework. When the TA finds a mistake in a program, a test shall be added, and all of them will go through the pipeline, and that becomes one less thing to check for.

To enforce good practices, the best I can think of is to make the application grow assignment after assignment. If it is poorly coded, expanding and maintaining it will be a major effort, and the benefit of good practices will become obvious. Building larger code bases is, in my opinion, the only way one can really understand and believe why patterns and conventions are useful and not just the product of some picky guys with OCD.

But you can cheat. For example, you can manually thoroughly check the submissions with lowest style scoring that pass all the tests (supposing they would be the ones more poorly coded), and find their corner cases where they would fail and add them to your battery of tests. This is unfair, though, as you are targeting your punishment to specific submissions.