We currently use an online quiz tool to give students questions in some of our engineering classes. These questions might be simple or very challenging, but at the end of all of them the students are required to put in "the answer" - a numeric value, or rarely a formula they've derived. These quizzes are run every week, and are basically a sneaky way of getting students to do homework problems. In that, they are very successful. The quizzes are basically open-everything - they can google things, talk amongst themselves, ask us anything etc.
However, we are becoming aware that this type of quiz leads students to race to find the right equation, plug in the right numbers, and move on. As an example, one of the questions from a recent circuits quiz was relatively straightforward in terms of the circuits concepts, but required careful planning of the math in order to avoid any errors. We've been tracking the questions that we are asked during the quiz, and almost universally the students 100% understood the circuits concepts but were making math errors. This is actually a good thing - we want to encourage students to learn to structure their reasoning in a careful, well-planned way, so we are OK with students struggling at it. However, the question doesn't allow for any way to give feedback on it. The answer either is or is not numerically correct.
Obviously we can have the students submit to us and we can evaluate them directly, but that isn't the point. An automatic grading system isn't ever going to be able to give students the same level of feedback as a human TA, but I feel that we are not using it to its fullest potential to give students feedback on their process.
What are some strategies we can use for question design in automatically graded quizzes that might help here?