31

Big collaboration projects such as LHC experiments (ATLAS, ALICE _e.g.) and LIGO result in academic papers featuring hundreds or even more than 1000 authors. Correct me if I’m wrong, but in fields like particle physics, researchers seem to belong to one of a small number of possibly competing collaboration projects. So if a paper resulting from a huge collaboration is submitted for peer review, the number of qualified referees would be very small (excluding researchers from competing collaboration projects), such that the anonymity of the referees would likely be compromised.

Is peer review still meaningful for papers resulted from huge collaboration? If yes, how should it be conducted?

Improve this question
4
  • 8
    This is an interesting question, and I am looking forward to the answers. Hopefully somebody from physics can chime in here.
    – xLeitix
    Feb 19, 2016 at 14:42
  • There are actually efforts underway to design high-sensitivity gravitational wave detectors on bases other than stabilized Michelson interferometers, but they are still in early days because they are meant to go space-borne. So there are people qualified to comment who are not in direct competition, as well as scientists from the other big interferometer experiments (who are in direct competition but really know what the you can expect from this class of instruments). I would guess that Phys. Rev. got one of each or something like that.
    – dmckee --- ex-moderator kitten
    Feb 19, 2016 at 23:57
  • You wouldn't necessarily exclude competing collaborations, since they're the best qualified to evaluate the work (and physics isn't that cutthroat). Actually with LIGO everyone was expecting the Virgo experts to be the referees, but then they joined together to submit the paper.
    – user4512
    Feb 20, 2016 at 6:59
  • Actually, having referees from competing experiments is THE norm. Who is better qualified than someone who is doing exactly the same thing, just with a slightly different equipment?
    – famargar
    Jan 3, 2018 at 11:58

2 Answers 2

Reset to default
25

While I am not certain about physics, I can speak for biology, which also has a large number of mega-author papers. There, the overall field is large enough and well enough connected that, although a massive number of biologists may have a conflict of interest, an even more massive number of biologists are not directly involved but still working in areas close enough to be good reviewers.

I also have a suspicion (though I cannot back this up with data) that mega-author papers may in general have an easier time in review than typical small-collaboration papers. Peer review evaluates papers along several different axes, key among which are presentation quality, significance of problem, validity of experimental method, and interpretation of results.

In the case of a massive collaboration, typically the questions of significance and validity of experimental method are long resolved: by the time the human genome was submitted for review, nobody was questioning its sequencing methods and a large scientific community was just waiting for enough data to accumulate; likewise for high-energy particle physics experiments. Similarly, no mega-author paper should be able to be submitted without enough eyes on it to ensure that the presentation is decent. That leaves interpretation of the results, and my understanding is that these mega-author papers tend to more focus on presenting data (which is already known to be of interest) rather than on its interpretation, and the interpretation (e.g., "What can we learn from the human genome?") is sorted out across many later publications.

Thus, review of a mega-author paper may actually be much easier than a small paper, simply by virtue of the difference in its goals and scale.

Improve this answer
9
  • 4
    Can we assume that a paper with 100+ authors was duly reviewed by those 100+ authors and several other colleagues before being submitted for peer-review, and that amount of different points-of-view from the people above already worked as a sort of (biased) peer-review?
    – Mindwin Remember Monica
    Feb 19, 2016 at 18:05
  • 2
    @Mindwin It's definitely not reasonable to assume that all of the authors reviewed it (unfortunately, that's not even reasonable to assume for a 5-author paper), but at least a significant fraction likely did. Thus, there is some degree of internal review, but so is there for any paper with co-authors.
    – jakebeal
    Feb 19, 2016 at 18:15
  • 7
    I hope that the order of magnitude is closer to a kilo-author than to a mega-author.
    – Federico Poloni
    Feb 19, 2016 at 18:20
  • 3
    I was once lead author for a moderately large collaboration on a fairly specific topic. Early in the process, one of my coauthors told me, "If you can get this thing past all the authors, peer review will be a breeze!" And she was right, to some extent. The paper had been seen by many more eyes in the early phases, so there were maybe fewer flaws for reviewers to find.
    – z_dood
    Feb 19, 2016 at 22:39
  • 5
    In the nuclear and particle physics collaboration I've been part of, our major papers typically went through an extensive sequence of internal consultations during which every member of the collaboration had the opportunity to review the entirety of the drafts. My very non-scientific polling suggests that perhaps 1/10-1/4 would actually read the whole document during any given round, many would read it at some point, and most would read the section(s) most closely related to their contributions. And we'd still get requests for clarification from the journal's peer reviewer.
    – dmckee --- ex-moderator kitten
    Feb 19, 2016 at 23:43
6

You are correct in implying that in these cases (few, very large experiments on the same topic), your peers are most likely also your competitors. However, when it comes to peer review, the issue is not one of competition, but rather one of bias. As some people have replied, there are extensive internal hierarchies and editorial committees within these experiments (I was in the ALICE experiment), which set the agenda and conduct internal review. The research is essentially reviewed by the time it reaches publication. However, it could suffer from bias, which external peer review may correct. The experiments at the LHC are very similar in their mission, but wildly different in their setup, so systematic errors and bias may be corrected.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .