Academia Stack Exchange is a question and answer site for academics and those enrolled in higher education. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Big collaboration projects such as LHC and LIGO result in academic papers featuring hundreds or even more than 1000 authors. Correct me if I'm wrong, but in fields like particle physics, researchers seem to belong to one of a small number of possibly competing collaboration projects. So if a paper resulting from a huge collaboration is submitted for peer-review, the number of qualified referees would be very small (excluding researchers from competing collaboration projects), such that the anonymity of the referees would likely be compromised.

My questions are: Is peer-review still meaningful for papers resulted from huge collaboration? If yes, how should it be conducted?

share|improve this question
5  
This is an interesting question, and I am looking forward to the answers. Hopefully somebody from physics can chime in here. – xLeitix Feb 19 at 14:42
    
There are actually efforts underway to design high-sensitivity gravitational wave detectors on bases other than stabilized Michelson interferometers, but they are still in early days because they are meant to go space-borne. So there are people qualified to comment who are not in direct competition, as well as scientists from the other big interferometer experiments (who are in direct competition but really know what the you can expect from this class of instruments). I would guess that Phys. Rev. got one of each or something like that. – dmckee Feb 19 at 23:57
    
You wouldn't necessarily exclude competing collaborations, since they're the best qualified to evaluate the work (and physics isn't that cutthroat). Actually with LIGO everyone was expecting the Virgo experts to be the referees, but then they joined together to submit the paper. – Chris White Feb 20 at 6:59

While I am not certain about physics, I can speak for biology, which also has a large number of mega-author papers. There, the overall field is large enough and well enough connected that, although a massive number of biologists may have a conflict of interest, an even more massive number of biologists are not directly involved but still working in areas close enough to be good reviewers.

I also have a suspicion (though I cannot back this up with data) that mega-author papers may in general have an easier time in review than typical small-collaboration papers. Peer review evaluates papers along several different axes, key among which are presentation quality, significance of problem, validity of experimental method, and interpretation of results.

In the case of a massive collaboration, typically the questions of significance and validity of experimental method are long resolved: by the time the human genome was submitted for review, nobody was questioning its sequencing methods and a large scientific community was just waiting for enough data to accumulate; likewise for high-energy particle physics experiments. Similarly, no mega-author paper should be able to be submitted without enough eyes on it to ensure that the presentation is decent. That leaves interpretation of the results, and my understanding is that these mega-author papers tend to more focus on presenting data (which is already known to be of interest) rather than on its interpretation, and the interpretation (e.g., "What can we learn from the human genome?") is sorted out across many later publications.

Thus, review of a mega-author paper may actually be much easier than a small paper, simply by virtue of the difference in its goals and scale.

share|improve this answer
4  
Can we assume that a paper with 100+ authors was duly reviewed by those 100+ authors and several other colleagues before being submitted for peer-review, and that amount of different points-of-view from the people above already worked as a sort of (biased) peer-review? – Mindwin Feb 19 at 18:05
1  
@Mindwin It's definitely not reasonable to assume that all of the authors reviewed it (unfortunately, that's not even reasonable to assume for a 5-author paper), but at least a significant fraction likely did. Thus, there is some degree of internal review, but so is there for any paper with co-authors. – jakebeal Feb 19 at 18:15
4  
I hope that the order of magnitude is closer to a kilo-author than to a mega-author. – Federico Poloni Feb 19 at 18:20
3  
I was once lead author for a moderately large collaboration on a fairly specific topic. Early in the process, one of my coauthors told me, "If you can get this thing past all the authors, peer review will be a breeze!" And she was right, to some extent. The paper had been seen by many more eyes in the early phases, so there were maybe fewer flaws for reviewers to find. – z_dood Feb 19 at 22:39
4  
In the nuclear and particle physics collaboration I've been part of, our major papers typically went through an extensive sequence of internal consultations during which every member of the collaboration had the opportunity to review the entirety of the drafts. My very non-scientific polling suggests that perhaps 1/10-1/4 would actually read the whole document during any given round, many would read it at some point, and most would read the section(s) most closely related to their contributions. And we'd still get requests for clarification from the journal's peer reviewer. – dmckee Feb 19 at 23:43

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.