2

Our School Research Ethics Committee requires that when designing an online survey that each question is optional for the participants. These rules were devised before the era of CrowdSourceing when researchers had to rely on the goodwill of others to gather the necessary responses.

However as I am paying each participant to complete this survey (using Amazon Mechanical Turk) do you think I have a case to argue that I should be able to make some, or even all, the questions mandatory?

I have not approached the Research Ethics Committee about this yet, I just wanted to see what the norm is in other countries and institutions.

Thanks

  • 6
    By "mandatory" you mean "if you don't answer this question you don't get paid"? Ethics aside, this may have an effect on your study's accuracy: if a participant encounters a question they don't want to answer, but they don't want to forfeit their money, they may just give a nonsense or random answer. If you make the questions optional, you can probably have more confidence that the questions which are answered, are answered accurately. – Nate Eldredge Jan 16 '15 at 17:45
3

Mandatory is a difficult concept. I think it is ethically questionable to not allow a participant to withdraw from a study at any time. There may be edge cases, for example drug studies, where further follow up treatment is required, but even here I think that treatment would not be considered part of the study. When a participant withdraws from a study we still need to provide them with whatever inconvenience allowance was agreed. We are not even allowed to prorate the inconvenience allowance, however, we break our longer studies into hour long segments and pay an hourly inconvenience allowance. It is perfectly reasonable to use a "forced choice" response system where participants are required to give a response in order to continue with the study. Using the Mechanical Turk, I think ethically you should provide an easy means of dropping out that is constantly available that will not penalize the participants. When it is so easy to withdraw, forcing them to answer a question which is not inherently necessary, seems silly. If they do not want to answer it and they have no other option, they will just withdraw.

Consider two experiments that consist of two questions. In the first experiment you either show either a circle or a square that is either red or blue. If for whatever reason the participant does not want to tell you what shape they saw, you can still get meaningful information about the color and therefore it would make no sense to force them to answer the first question. In the second experiment you show them a face and ask if it is a man or a women and based on that response you ask them if it was either Alice or Carol or Bob or Dave. For this experiment they need to answer the first question.

| improve this answer | |
  • I like your suggestion. A "forced choice" response system where any mandatory questions have an N/A option or an option saying they don't want to answer. I will also add an easy and obvious means of dropping out that is available on each page. Participants will of course be clearly informed before starting the experiment/survey that certain questions are mandatory but that they will have the option of stating that its either non applicable or that they don't want to answer. Partially completed responses are discarded of course and the participants have to be made aware of this before starting – Deepend Jan 17 '15 at 16:14
2

The IRB that I serve on would never approve of a paid participation survey in which the participants would be denied the compensation if they didn't complete the survey. Furthermore, if someone decides that they don't want to complete their participation in the study it would be unethical to make use of any partial responses that you'd already gotten from that participant.

In short, the answer to your question is "no."

| improve this answer | |
2

While this will definitely vary by review board, my personal experience has been that it's a standard that you are explicitly required to inform participants at an appropriate point (disclosures, usually) that they are allowed to refuse to answer any question they want by leaving it blank or marking it in some fashion. Sometimes this even means that, if they leave any question off, you have to throw out their whole set of responses because it's useless to you, but they must still be paid/credited for their participation even if it makes it useless to you.

The question is both ethical and practical - if you require an answer you may also be encouraging dishonesty, and possibly invalidating the usefulness of your survey anyway. "I'm not answering that" is better than "I'm not comfortable answering that truthfully, so I'll just put down nonsense." And this set of rules actually goes far beyond just the idea of unpaid volunteers, as it applies to things like medical patients, paid employees, etc.

Now, this is not to say you can't do it - ethics review boards have approved electrocuting and burning people based upon appropriate balancing factors, so it's up to the board and how you design and present your study.

But setting form fields as "required" will require appropriate disclosure to the participants and review board, and will likely be considered a negative that must be appropriately justified and handled. You will have to show that autonomy, right to withdraw, informed consent, non-coercion, and ethical payment are all handled appropriately. You will be making things hard on yourself, so if you don't have to do it this way, you probably shouldn't.

You are generally welcomed to pro-rate payment, so if a person refuses to answer all items you only have to pay them for the items they did answer - and you must put this in the informed consent forms and present this to the review board. Whether or not the particular online payment system or setup you use will support such a method might restrict your realistic study design, but that hardship is probably not something the review board will be particularly concerned about.

| improve this answer | |
  • Some very good points. I will have to look into whether or not it is possible to provide partial payment using AMT. – Deepend Jan 17 '15 at 16:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.