Do most journals keep archives of referee reports and would historians of science be able to access them? Perhaps after some delay?
-
2I would imagine that most journals these days keep archives. Among other things it helps with potential future retraction investigations. On the other hand, the archives are not necessarily complete. For example, the archives of Physical Review apparently don't contain any editor-referee communication from before 1938.– AnyonSep 16, 2019 at 19:40
-
I suspect, but don't know, that most publishers would prefer not to make these available for a variety of reasons up to and including potential liability. There are privacy and confidentiality concerns, competitive positioning concerns, etc. Just not embarrassing a reviewer for the occasional mis-statement is a consideration. I think it might take an extraordinary process to pry them loose. Perhaps highly sanitized versions might be made available, but not be very valuable for historical research.– BuffySep 16, 2019 at 19:52
-
1I think this question is a duplicate, but I can't find specific enough search terms to locate the earlier one. Someone else may have more inspiration than me.– Peter TaylorSep 16, 2019 at 22:29
-
See here for a case where the veil of referee anonymity was pierced, in order to provide insight into what happened in a historically important case: physicstoday.scitation.org/doi/10.1063/1.2117822 However, revealing in the information was considered a big deal. As the article states: "Blume’s release of the logbook records—a decision made because 69 years have passed and no one involved is still living—confirms the identity."– BuzzSep 18, 2019 at 22:51
Add a comment
|
5 Answers
All reputable journals today use web-based manuscript management systems that keep information about each manuscript, its various revisions, who was asked to review and what their reviews were, as well as editor decisions. So yes, this information exists in these systems at least as far back as the introduction of these systems (at least ten years ago). As the editor-in-chief of a journal, I can look at all of this data for every article in my journal.
But I suspect that it will be exceptionally difficult to get at this kind of information if you're not the editor-in-chief yourself. To do research on this means that you're doing research on human subjects, with all that implies: You'll have to have IRB [1] approval for your study protocol, you might have to anonymize the data (which I suspect would be exceptionally difficult in itself), and/or you might have to receive informed consent from the reviewers, authors, and editors (all of which are involved in the decision making process).
I have a suspicion that few publishers are interested in going to this level of trouble. On the other hand, I have no doubt that such studies have been done, so I would expect that it is possible to get at the kind of data you're looking for -- it's just very difficult to do this kind of research.
[1] In the US, IRBs (Institutional Review Boards) are the bodies tasked with overseeing research that involves human subject research. All human subject research has to be approved by an IRB before the data so gained can be used for publications.
answered Sep 16, 2019 at 22:58
-
2
-
3@Buzz That is strongly not true in my location, in the US. Consent may not be required by the IRB, but that doesn't mean IRB approval isn't necessary.– Bryan Krause ♦Sep 18, 2019 at 23:01
-
2@Buzz The names may not be necessary for the reviews to be identifiable. The IRB likely decides whether or not it needs full review.– Bryan Krause ♦Sep 19, 2019 at 1:06
-
2@Buzz Bryan is correct, you need IRB approval to be exempt from IRB approval. Sep 19, 2019 at 11:15
-
2I'll second @AnonymousPhysicist's comment: Research that is in the "exempt" category still requires IRB to determine that it is exempt. It's not up to an individual researcher to claim that it is exempt. Sep 19, 2019 at 15:46
Requests to review for The Astrophysical Journal, published by IOP Science, come with the following note:
"7. If qualified historians wish to use your report, we will ask you or your heirs. If you wish us to destroy your report, then please inform us."
I also have a vague recollection of a request for my consent to release my reports after a suitable delay (20 years?) for historical research, but I can't find the message, or recall which journal & publisher it was.
Many journals use open peer review - the reviews are published alongside the paper. See e.g. Biomedical Central, BMJ Open.
So this isn’t just for future historians - you can see the peer review comments now.
(The weakness here is you can only see reports for accepted papers - the rejected papers aren’t published, so neither are the reviewer reports)
While the data probably exists, referees in an anonymous review process are entitled to anonymity. I suspect that journals would not release the data out of concerns for the reputation of their processes.
They may be more willing for historically important papers when all concerned are no longer with us.
answered Sep 17, 2019 at 13:07
Seems to me that - as alluded to in some other answers - the big question is over anonymity of reviewers and confidentiality of the review reports. I think that any historical study would have to address these issues. The two likely ways of doing that, off the top of my head, are for,
a) Studies of historical science, where everybody concerned is dead. Probably not enough time has gone by since review systems were computerised for this to be relevant yet.
b) Statistical studies, where robust efforts have been made to anonymise and aggregate the data.
