9

The incoming editor of Psychological Science mentions in an interview that pre-registration isn't feasible for all submissions to the journal:

In instances in which preregistration is feasible... we will continue to encourage authors to take these steps. Yet we must be fully cognizant that in some areas of the broad discipline of psychological science, these actions may not be possible or may be ill-advised.

What are examples or types of experiments or studies (in psychology or elsewhere) where pre-registration wouldn't be possible or would be ill-advised?

16

Examples are areas where a study takes advantage of a sudden, unexpected event. For example, there have been studies about human memory related to unexpected tragedies. There was an idea that lots of Americans remembered exactly where they were when they found out that President Kennedy was shot. After the Challenger disaster, a study was done where they asked people where they were when they found out, and then asked them different amounts of time later (e.g. a year, 5 years and then ten years out), and they found that despite people being very certain of their memories, many people had recollections which changed drastically in the intervening time. Similar studies were done later for the Columbia disaster and 9/11. See summary [here], and 10 year followup to 9/11 [here]. These would be extremely hard to preregister without substantial ethical problems; I suspect that most journals and the public would frown on psychologists deliberately causing large-scale national catastrophes in order to preregister their studies.

The World Trade Center attack also provides also an example from another field: Right after 9/11 airplanes were grounded throughout the US. Multiple studies used this as an opportunity to study the impact of airplanes on the atmosphere. See here.

Editorial remark: The set of studies where preregistration is not feasible is a very tiny fraction of all studies, and the fact that such studies do exist should not in general be taken as an argument against broad requirements for preregistration.

| improve this answer | |
  • 2
    While one of course can't register a study for a future disaster on a specific date, it would seem to me possible either to register a study for a not-yet-specified disaster and date (giving a specification for what kind of disasters would qualify) or at least to register at the start of the study, before significant (or ideally, any) investigative work has been done. Do you have any thoughts on this? – cjs Sep 13 '19 at 11:41
  • @CurtJ.Sampson That certainly seems plausible, but I'm not an expert. Obvious issues would be that any disaster that meets the criteria might be too rare for this to be practical. As for the second suggestion of partial preregistration, that seems plausible, but my impression is that this class of study requires extremely fast reaction time already for both logistics and IRB reasons, so even some form of partial preregistration at the start would pose a potentially significant additional burden. – JoshuaZ Sep 13 '19 at 12:07
  • 1
    As a complete aside, there is a quite good episode of Malcolm Gladwell's "Revisionist History" about these memory studies. – Wolfgang Bangerth Sep 16 '19 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.