1

As an example, imagine you're running the faculty of computer science and decide which books to use for the next semester.

  • You pick book A for all students -> no IRB review needed
  • You pick book B for all students -> no IRB needed
  • You pick book A for half the classes and book B for the other half (due to professor preference, for example) -> no IRB needed
  • You decide to run an experiment where you want to see whether book A works better than book B -> IRB needed
  • You retroactively decide to analyze whether book A was better than book B -> simplified version of IRB is needed for retrospective data analysis

What is so special about something being an experiment?

9
  • 2
    I believe your prem8se is wrong, at least in the US. It's not research unless you plan to disseminate the results. Commented Jan 3 at 17:01
  • 4
    I suspect what triggers the need for an IRB is not the fact that it's an experiment, but the fact that it's human subjects research, with your students being the subjects. Commented Jan 3 at 17:08
  • 1
    Non-experiment research is also regulated. "Experiment" is not the relevant feature, research is
    – Bryan Krause
    Commented Jan 3 at 17:10
  • 5
    @JonathanReez - and now you have a useless "experiment" since you can't separate out the profs from the books.
    – Jon Custer
    Commented Jan 3 at 18:05
  • 1
    Existing methods exist for Textbook evaluation. Doing legwork and a quick search to help inform this question would help to improve the question. For example, I discovered project2061.org/publications/textbook/articles/approach.htm for the US that covers the example in the question: How to evaluate textbooks. Also, many papers exist on using teaching evaluations for research. Lastly, the OP could improve their post by citing sources for their definitions or "research" and "experiment" Commented Jan 3 at 21:11

5 Answers 5

8

Research creates a conflict of interest between the researcher and the research subjects: researchers benefit personally in their profession for performing and disseminating research. Research protections exist because people abused their research subjects for these personal gains (money, fame, etc), they weren't invented to create paperwork.

Seemingly innocuous research is still regulated or at least subject to institutional review because it's not sensible to have the researchers who benefit from their research make their own decisions about what they think the risks are or what paperwork they want to do.

Many things are regulated in one context but not another; research is not particularly special in that sense (for example: if friends come to your house, you can make them dinner and serve them beer without any sort of permit; if you want to collect money for making dinner and serving beer you typically need a permit for each and are subject to many additional regulations). For this to work, it's necessary to define what is and isn't regulated somehow. The standard I'm most familiar with is that research is intended to contribute to generalizable knowledge. It would not be practical or reasonable to regulate an instructor's ordinary choice of textbook as if it's research any more than an ordinary medical procedure.

11
  • 3
    @JonathanReez Yes that's exactly what I'm saying. We don't regulate all decisions as research, we regulate research.
    – Bryan Krause
    Commented Jan 3 at 20:25
  • 1
    @JonathanReez We don't merely “assume” KB-1 and KB-2 are equivalent. Patients expect and physicians have a duty not to carelessly chose a treatment that's inferior in every way. If the physician choses a treatment for any other reason, this expectation breaks down. In fact, as an experimenter, it's very tempting to chose treatments that you expect to be different to maximise statistical power and make it easier to publish. That's one reason why intent matters (and disingenuously omitting to “label” it as an experiment – as you put it in your question – doesn't really change anything).
    – Relaxed
    Commented Jan 3 at 20:29
  • 2
    @JonathanReez But not all else is equal, which I discuss in this answer.
    – Bryan Krause
    Commented Jan 3 at 20:40
  • 1
    @JonathanReez You're describing things for an IRB to consider when evaluating a research proposal. Having researchers decide their own work is low risk would be like having the police investigate themselves for possible wrongdoing in policing.
    – Bryan Krause
    Commented Jan 4 at 1:36
  • 1
    @JonathanReez You're the one who contrived an example with two equivalent treatments but ultimately that's completely irrelevant, the point was about how you approach the treatment decision for an individual patient, whether you are only trying to do what's best for them in their specific situation or if other considerations figure in your choice.
    – Relaxed
    Commented Jan 4 at 10:42
3

To be specific to your scenarios:

In the US, the controlling department is Health and Human Services, and their requirements are easily found at HHS.gov. From "§46.104 Exempt research." subpart (d) we find

(d) Except as described in paragraph (a) of this section, the following categories of human subjects research are exempt from this policy:

(1) Research, conducted in established or commonly accepted educational settings, that specifically involves normal educational practices that are not likely to adversely impact students’ opportunity to learn required educational content or the assessment of educators who provide instruction. This includes most research on regular and special education instructional strategies, and research on the effectiveness of or the comparison among instructional techniques, curricula, or classroom management methods.

So, in the US at least, no IRB review is required in any of your scenarios above.

3
  • Good answer. As an aside, I've worked with academics who wanted to use course evaluations for research and their specific universities still required IRB approval. Likely specific universities erring on the side of being risk adverse given your references. Commented Jan 3 at 21:05
  • 1
    @RichardErickson - that is a grey area, because you are now (potentially) collecting information from individuals ("interaction" or "intervention" in the definitions section). I would bet the IRB would go very smoothly, particularly if the course evaluation questions were the standard set used by the institution already. Asking specific, course-related questions could be seen as different.
    – Jon Custer
    Commented Jan 3 at 22:25
  • 1
    The professor I worked with wanted to evaluate her teaching methods using research and you're correct. She got IRB approval prior to teaching and had no problems. Sometime real life human interactions go smoother than hypothetical internet questions :) Commented Jan 4 at 1:43
1

What is so special about something being an experiment?

The answer is location specific for exact policy, but largely depends upon what you want to do with the results. For example, Loyola notes

Do student course evaluations of faculty need to be reviewed by the IRB?

Student course evaluations of faculty do not need to be reviewed by the IRB, unless the evaluation results will be used for research purposes or will be published or disseminated.

Basically, if you want to do (including the important step of publishing) research, you need IRB approval. This example would directly apply to your example as well.

6
  • 2
    What makes you think it's not acceptable. It is perfectly acceptable, it just undergoes one layer of review to certify it is compliant.
    – R1NaNo
    Commented Jan 3 at 17:33
  • 3
    @JonathanReez -- This is because the IRB exists -- in the US, by Federal Mandate -- to review the treatment of human subjects research, not teaching. Students, just by being students and participating in coursework, are not "research subjects". There are certainly ways to abuse your students while teaching, but they are beyond the purview if the IRB (though many of them are covered by different laws, federal or otherwise) Commented Jan 3 at 17:47
  • 3
    A necessary element of research is 'generalizable knowledge.' If your purpose is to develop a conclusion that can be widely generalized to the pedagogic process then you are seeking generalizable knowledge and you must have IRB review. If you just want to choose a better book for yourself, you don't. Commented Jan 3 at 18:06
  • 1
    Anecdotally, some psychologist at my school was doing a study surrounding course evaluations, and a different evaluation was used for a subset of courses. The questions weren't shown to faculty. Some of the students were not comfortable with the questions. Students and faculty in the involved courses got an apology from the Dean and Provost, stating that the IRB had retroactively determined that IRB review was required to roll this out (I can't remember if they also said informed consent was required). I know this, as I was teaching one of the courses involved, and got the apology letter. Commented Jan 3 at 18:14
  • 1
    @ScottSeidman yep - but if the same questions were shown without the intent of doing "research", its likely that no one would take the complaints seriously or apologize for anything. Commented Jan 3 at 20:22
1

If you are conducting/disseminating research related to human subjects, you will need IRB approval. This is a safe-guard. Just switching books and keeping note internally is part of departmental pedagogy and something those students signed up for, being the subject of research is not (unless you allowed them to sign a disclaimer, which itself would require IRB approval).

It's not the experiment that's the trigger, it's the human subjects.

In a case like this your IRB approval will just sail through. Someone looking at efficacy of pedagogical material is not high on the list of the type of ethical violations universities/agencies are trying to curtail.

EDIT: I see from your other comments that you are looking at the requirement for IRB approval as meaning that the work or topic is not acceptable. It is perfectly acceptable. It just requires one layer of overview/review to ensure compliance. That's all.

Not the best analogy, but on the fly I see it somewhat similar to when you order something from a foreign vendor, your credit card may need an authentication code to make sure there's no fraud. It doesn't mean you can't buy it, just one layer of protection for everyone involved.

8
  • I disagree with your characterization, at least in the US. It's not just the Human Subjects, it's the subjects and the RESEARCH, and research is "a systematic investigation designed to develop or contribute to generalizable knowledge relating broadly to public health, including behavioral and social-sciences research." (from ncbi.nlm.nih.gov/books/NBK22941) -- no desire to contribute to the generalizable knowledge (i.e., no publication, or plan to do so) == not research. Commented Jan 3 at 17:42
  • @ScottSeidman my response was implicit that it was research related to Human Subjects which triggers an institutional review. That seemed to be the crux of the question from the OP, ie what is triggering IRB, not necessarily all of the nuances of IRBs. Perhaps I misunderstood.
    – R1NaNo
    Commented Jan 3 at 17:46
  • @ScottSeidman That definition, though, would suggest that participant observation/action research/qualitative-methodology research projects, where generalizability is not among the goals, would not require IRB approval, but I bet they do. Commented Jan 3 at 17:55
  • 2
    @DanielHatton I suppose different IRBs can be more stringent, but the definition comes right from the Code for Federal Regulations. I can tell you that I've gone to my IRB about 20 times with questions like this, and every time, the question I get back is "do you intend to publish?", and if the answer is "no" their response is "you don't need IRB approval because it's not research". Commented Jan 3 at 18:02
  • 1
    I don't think this study would necessarily "sail through" an IRB. There is no described way for a student to opt out of the study. There are classes of studies that don't require this, but in this case, there is conceivable harm to subjects (e.g., if students with textbook A do great in the course, but students with textbook B all fail, then the latter group has been harmed). Commented Jan 3 at 18:09
1

While I don't believe you posit research here, research subject are protected, often with IRB review being a part of that process, because horrible injustices have been done to subjects in the guise of research.

When dealing with students, in particular, some of the main risks have to do with coercion and voluntariness. Also, in the US, subjects have a right to NOT PARTICIPATE in research, and the IRB would look to make sure you have a fair path around the research for student who opt not to participate.

Also, students who are research subjects also need to have their privacy protected, and they need to be protected from social wrongs.

But again, I disagree that the activity in your description, in and of itself, constitutes research under US PHS definitions.

2
  • If I publish a paper comparing student performance with Book A vs performance with Book B, can I do so without IRB approval? Commented Jan 3 at 17:22
  • 1
    @JonathanReez No, you cannot -- but it's the publishing (or plan to publish, anyway) that makes the difference, not the assessing textbook efficacy for your own teaching purposes. Commented Jan 3 at 17:36

Not the answer you're looking for? Browse other questions tagged .