3

My colleagues argue that a meta-analysis / systematic review needs at least two data collectors (or authors according to them) as a "must". I have not found any reference in the PRISMA (or any other sources about meta-analyses/systematic reviews) that confirms this claim.

Do you know any such protocol or consensus?

  • 1
    It seems to me I am missing a lot of context for this question. For instance, what field are you talking about? – xLeitix Jan 5 '14 at 0:48
  • Thanks for the input. I asked my question as a completely general question regardless of fields, as e.g., PRISMA is already regardless of fields. – Vic Jan 5 '14 at 0:50
4

There is no such requirement. It is frequently helpful to have multiple authors to help sort through the vast amounts of literature being covered, but is by no means a necessity. For instance, a colleague of mine recently published a fairly substantial review article on his own.

However, in medical research, a meta-review serves a rather different purpose than the usual review article in other fields: it is collecting the results of a bunch of different experimental studies, and trying to reach an overarching medical recommendation. That's a rather different research function than a review article in other fields. In medicine, you would want to have multiple people reviewing the data to make sure that it's not one person unilaterally deciding everything independently.

  • This is what I say to my colleagues. However, I think you are confusing a simple review with a systematic one. Besides, there is a database in medical research which collects the systematic reviews, and that very database (Cochrane) encourages more than 1 observer in one of its resources, and does not say anything about it in another source it publishes (meaning that it sorta contradicts itself). – Vic Jan 6 '14 at 0:04
  • I've updated my answer. Medicine is not the same as other fields. – aeismail Jan 6 '14 at 0:11
  • Thanks. I should add to your new response that in medicine too there are many single-observer systematic reviews, and it is not a "must" according to any authorities. But some colleagues incorrectly argue that it is a must. I am trying to prevent a wrong message. – Vic Jan 6 '14 at 0:20
  • Not a must—but almost certainly a "really, really should!" – aeismail Jan 6 '14 at 0:25
  • according to whom? Can you name an authority? or are you explaining your "own" personal view? These two hugely differ, and we should avoid practicing one when sounding like the other one. "but almost certainly a "really, really should!" " sentence does sound like you are a director member of Cochrane ;-) or some other authorities. Maybe you can be a journal chief editor, but that does not suffice to infer your own personal view as a globally accepted consensus (or at least to sound like an authority). :-) – Vic Jan 6 '14 at 0:52
3

From the perspective of an Epidemiologist, with some published meta-analysis experience:

Must is a very strong word. Some people have pointed to where, for example, the Cochrane Collaboration requires it, and PRISMA might not, but generally speaking I've never really encountered a situation where having a single author on a meta-analysis was a substantial barrier to publishing a review.

On the other hand, I would say that a meta-analysis or systematic review should have a second person on the study team. Rarely is the literature being reviewed so clear, so well-laid out and so utterly free of ambiguity that a single person can read, digest and abstract the literature without making any judgement calls. Without having any papers where they search and simply cannot find what they're looking for. Without hitting that one paper they simply cannot make heads or tails of. There should be someone there to double-check your work, or a sample of it at least, to make sure what you described as the system in your paper and what you actually did match up. To look over those papers you've set aside in the "Problem" pile to see if they can see things you don't.

I leave whether or not that person should be an author as an exercise to the reader.

I know some colleagues who essentially begin all reviews with a parallel, blinded double-abstraction of the papers once they've been found - or even begin all the way at the search being carried out twice. I don't know that I'd go that far, but it is extremely helpful to have someone to double-check your decision making against. Despite being careful, reading closely and reviewing my own decisions, I have yet to work on a meta-analysis where I haven't been glad to have a second reviewer (or to be said reviewer).

  • Thanks. You outlined what was in my mind very clearly. Unfortunately, in the field I publish, people are not usually familiar with things such as statistics or different types of review. So I have seen reviewers who throw recommendations they really should be silent about (since they know nothing about) and editors who are not competent enough to decide what is right and what is wrong. So I have seen such issues. I agree two or even more parallel observers "might" (might) favor the reliability, but I am also aware of the bias it introduces as well as the false sense of safety that can lead – Vic Jan 6 '14 at 20:18
  • to potential errors (since each observer might become less careful, thinking that the other one is already doing the job)... Besides, a single observer can always double check the work. – Vic Jan 6 '14 at 20:29
  • @Vic - All I will say is that I've been under the impression I've double and triple-checked, only to have something found by a second reviewer. – Fomite Jan 7 '14 at 18:13
1

Answer: Although PRISMA does not require such a protocol, Cochrane Handbook does. However, it too does not "necessitate" it, but encourages it (at least to my understanding).

Check out the Chapter 7 of the Part 2 of this online Handbook: http://handbook.cochrane.org/

  • Sure, added to the post. – Vic Jan 5 '14 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.