It is ethical so long as problem sets are not a substantial part of the final course grade. The pedagogical value can be increased, and concerns about unfairness mitigated, through some minor adjustments.
I use a version of this in some of my courses. The major concern that you raise is one of unfairness due to the randomness of which problems are evaluated.
The ethics of the assignment should be evaluated with an understanding of the difference between formative and summative assessment. Formative assessments are supposed to provide feedback to the student and the instructor about the student's (and the class's) progress in the material. Summative assessments are supposed to provide an evaluation of student mastery at the end of a course module or entire course.
Summative assessments are high stakes (i.e., have a substantial impact on a course grade) and can include things like term papers and examinations. Randomness itself in a graded assignment is not a concern, but would become a concern for a high stakes assessment. An example where I think it is handled well is that, as an undergrad, I often encountered essay exams (usually in humanities or social science courses) where we were given 5 questions to prepare for, but only 3 appeared on a final exam (and sometimes the student would only have to answer a subset, like 2 out of 3). This offers a nice balance between having an overly long exam vs. a short exam where the students study to the test, i.e. ignore material that will not be on the exam. An extreme in which it is handled poorly would be assigning three papers and grading only one of them, where the final paper represented 30-50% of a final course grade. This would be indefensibly arbitrary, and I've never seen anything like it done.
Formative assessments should be low stakes, and some instructors seem to think should be no stakes (i.e., not graded at all). The question is how do we treat problem sets, which are common in STEM courses where practice of methods is necessary to learning.
Problem sets sit in an awkward place. They are often graded, and sometimes form a substantial part of a course grade, yet their place in a course sequence really means they should be treated as a formative assessment:
Reading/Lecture → Problem Sets → Examinations
In some sense, if the student does well on the summative assessments, the formative assessments shouldn't drag their grade down. That is, if a strong student is lackadaisical about homework but aces the exams, giving the student a poor grade due to missing or sloppy homeworks seems both punitive and just plain inaccurate as an assessment of their capabilities. So if their purpose is formative rather than summative, problem sets could be ungraded, and used entirely to provide practice for the student and feedback on how well they are learning the material. Unfortunately, I have found that if the problem sets are ungraded many students won't do them, and subsequently will perform poorly on exams.
Sidebar: Doing the problem sets improves student performance, but inevitably there seems to be a high correlation between the students who don't do the problem sets and those who I suspect would struggle with the material no matter what. I think the causality is that students who are underprepared get discouraged by the problem sets, and then don't do the work they need to in order to master the material! Trying to encourage those students while still making it clear that they have to actually do the work is a constant struggle, and probably not unique to me.
The students have to do the problem sets in order to learn the material, but the grades are really utterly irrelevant to the assignment's pedagogical purpose. But getting the students to engage with the material is very difficult if they are no stakes assignments. Offering low stakes (maybe all the problem sets are no more than 10% of the final grade in total) is probably enough motivation to get them to do the work. Since the purpose is practice rather than summative assessment, assigning more work than will be graded is acceptable. It becomes less acceptable if the impact on the grade is higher. It also becomes less acceptable if nothing is done with the ungraded problems. That is, you shouldn't just grade some of them and completely ignore the ungraded ones. Ungraded problems can be gone over in class, in discussion section, or answers could just be provided for the students to self-check.
Finally, there are a number of ways that the assignment could be changed to increase the pedagogical value.
- All problem sets can be graded for extra credit. I do this in some of my courses. I have found that it provides sufficient motivation for most of the students to attempt the problem sets, while mitigating student concerns regarding grading of random answers. (If they don't earn the points, it's "just" extra credit, and has no impact if they do well on other assignments.)
- For your example, points can be awarded for 3 out of 4 possible answers. That way, if a student skips (or just messes up) one question out of the 4, that one will be ignored. Points will be awarded only for best 3.
- Switch to the flipped classroom model. This is of course a major pedagogical shift. In this case, the problem sets would be done in class. They could be completely ungraded, as the professor and teaching assistants will be observing the student learning directly (both that they are doing the work, and how well they are doing). Or they could be graded, but students will have assistance with them. Or they could be marked "complete" if the student is present and works diligently, rather than for correctness, but the professor could require them to be submitted for a grade by students who are absent or clearly not doing the work while in class.
Any of these things would improve the pedagogical value of the problem sets. But the randomness in and of itself is not an ethical concern so long as the problem sets are not a substantial part of the final course grade.