What does a systematic review entail? In what way is it "tedious"?

Question

I spoke with a research professor who informed me that I may be able to work with him on a systematic review. However, he warned that it can be very tedious and frustrating so I should think about it. I understand that a systematic review involves looking through many papers, however, I would appreciate if someone could tell me, from their experience, how it is tedious. I would also have other classes in addition to this research so would be able to work on it for about 10 hours a week.

Answer 1

Tedious has a negative connotation. In many ways systematic review is meticulous, which to some people may appear tedious. One who enjoys reading, inspecting clues, and putting puzzles together to reveal hidden patterns would enjoy doing a systematic review.

The first hurdle is perhaps setting up the research question and search criteria. If the set of keywords is not close to perfect the resultant articles may not be exhaustive, and being exhaustive is a spirit of systematic review. In addition, all searches need to be documented through writing or using codes, so that the search can be replicated by other people.

The second hurdle is combing through moderate to large amounts of abstracts in order to identify the right targets. A set of carefully developed selection criteria will help. But a good handful of the retrieved abstracts may still fall into the gray zone and require some meetings to sort them out

Then it comes the actual reading and data extraction. This can take months and months of reading, re-reading, tabulation, and revision. Again, if the authors know what to extract very clearly, the process can get less iterative.

Notice that for the previous two hurdles, some research team may require two or more researchers to do them independently and then compare their results for agreement before divvying up the work. Be prepared for frequent team meetings or communications for troubleshooting.

The last hurdle is to put all the tabulated information into use, through grouping, sorting, summary, and synthesis. This requires a great deal of understanding from the reading process and one's experience. The process can often be iterative. Making a good conceptual framework will help setting up a more robust draft. Keeping the research questions in mind as the north pointer can also prevent getting lost.

And, all of the actions above need to be documented so that anyone can pick up the review and conduct a similar search (say, maybe 5 or 10 years later) in order to compare with what you found. In a way, systematic review can be viewed as an exhaustive, planned, and replicable literature review.

Now back to your case. If it's 10 hours a week for a semester, chance is you will not see the fruition of the article, but you can definitely take part. The important part is to get a very clear idea which part of the process you're taking. Article searching and downloading may work for more flexible schedule, even 10 hours on a day per week is fine as long as you keep good searching notes. But if it's the later stage, it's better to work on it by small amount, like 2 hours every day, as the later stage requires more consistent immersion and regurgitating.

And if I may make another suggestion, ask if the supervisor needs anyone to do a more general literature review for a grant proposal or a journal article. That should give you a smaller, more defined task and you can use this chance to see if you like this kind of work and, more importantly, if you and your supervisor work well together.

Answer 2

A review article can be a massive undertaking.

There might be no co-authorship if your contribution is not significant enough, if the scope of the review paper is very large and your contribution is a very small proportion of the work involved. Check whether or not you may expect authorship credit.

I am concerned that this might end up being grunt work for you.

On the other hand, even without co-authorship, if you are really interested in the subject, and if you are able to put clear limits on how much time you put in, and stick to them, it could be worthwhile for you.

It can be tedious in the sense that there can be so many articles to include in the review that it starts to feel like you're crossing the Atlantic Ocean in a rowboat.

(Not that I have ever written one. But I typed one in LaTeX for my advisor and even that started to feel rather endless after a while. I am still astounded how he managed to keep track of so many different papers.)

Answer 3

tedious and frustrating

Systematic reviews are tedious and frustrating. There is a high level of transparency and accountability in every stage of the review process, which can make it very complex. Many "big" reviews are a product of a series of articles as the result. Some reviews publish there methodology as a separate article, BMJ Open is one I have seen, and then their review in another.

Ultimately, you are setting up and defending a process that another researcher can hopefully pick up, rinse and repeat when they want to pick up new articles from when your review finished. Setting up a search strategy, usually with an academic librarian is important. Then strategising and documenting the tedious process before the next step of sifting through the sometimes enormous numbers of articles. I have seen reviews which included nearly 10K to 20K title elimination. That is probably where the frustration levels hit the roof (and you haven't even started the analysis and write up process!).

I thought this is worth a new contribution because there are now plenty of systematic review tools that can help reduce the tediousness and frustration. So more time can be spent on the "meatier" sections of analysis and write up, but there is no consensus on the tools as yet.Most of the systematic review tools are developed by the health disciplines, so understandably some are open source are available to all. So you can hopefully adapt them to whatever field that you are in. My simple Google found these sites, but I am sure you can find more appropriate ones for yourself (Google adapts to your preferences).

Personally, I was keen to use Covidence for my review, but my supervisors were keen to stick to their tried and trusted Excel, argh.

https://ktdrr.org/resources/sr-resources/tools.html

http://systematicreviewtools.com