I have thought about setting up a quiz as part of a job application form (see background below for details), where I ask about 5 multiple-choice questions which are trivial to answer (or require a minimal Google search) for people in the right field, no more than 2-3 minutes altogether. These would require a significant amount of effort to answer for people without a matching background for the position, since they would need to Google around for a while until they find the correct answer. The application submission will be possible only to those whose answers are correct, effectively blocking many of the applicants with a background mismatch who do not want to spend this much time to apply for a position they're not qualified for (at least this is my hypothesis). That is, the job application form can only be submitted if the quiz answers are correct, in addition to having completed all other requirements (name, current institution, referees, etc.). Otherwise the form will return an error until all the answers are right.
I would like to know what are possible issues I should consider with this system logistically and, mostly, ethically (I cannot really think of ethical issues here, but that's why I'm asking, to make sure). I would also like to know whether I should expect some of the relevant candidates (i.e., those with a matching background) to get annoyed by this system.
Every time I advertise a position (at the postdoc or PhD student levels) I get lots of applicants whose background doesn't match the requirements (with wide ranges of mismatch, from wrong subfield to wrong field). Some times this is obvious from the beginning, some times I need to read the application materials in more detail. These applicants often just copy-paste their application materials from some previous call, which generally means they put little effort into it.
I have a system in place to try and minimize the amount of spam/wrong background applications I get. Essentially, there are text fields where the applicants need to insert information, like a description of how they contributed to up to 5 publications. I use this system because 1) the effort is bigger than just attaching a precompiled PDF and deters the laziest applicants and 2) it is easy to quickly spot the most uninterested/unqualified applicants because they tend to leave the 5-publication fields empty.
Still, many bad applications make it through the system, and I end up spending lots of time sorting through applications with approximately zero chance of success. If I spend 10-15 minutes per bad application, going through it plus sending a rejection email to the applicant, it's still a significant amount of time (let's say I have to go through 20-30 of these, that means between half and a full day of work wasted on this, which I could have spent helping my group members with their research).
I want to avoid all of this wasted effort, since the situation is unfair to everyone involved, including those candidates with a matching background whose applications I need to review in less detail than I would like because those spam applications are sucking time away from it (I would say, for any given opening, I usually receive only about 30-40% of serious applications, meaning all strong and weak candidates with the correct background).