Medical Sciences: Systematic Review Overview

Key components of a systematic review include:

• Systematic and extensive searches to identify all the relevant published and unpublished literature.
• Study selection according to predefined eligibility criteria
• Assessment of the risk of bias for included studies
• Presentation of the findings in an independent and impartial manner
• Discussion of the limitations of the evidence and of the review.

Systematic reviews can evaluate a range of evidence; qualitative, quantitative or both. Appropriate methods of synthesis should be used for different types of evidence.

The systematic review process has been developed to minimize bias and ensure transparency. Methods should be adequately documented so that they can be replicated.

When conducted well, systematic reviews should give us the best possible estimate of any true effect. An assessment of the methodological quality of reviews should highlight the limitations of a review.

Understanding and interpreting research evidence is an important part of practicing evidence-informed public health. This short video from the National Collaborating Centre for Methods and Tools explains the types of reviews and what kind are needed.

Time: On average, systematic reviews require 18 months of preparation.

…to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months…” (Cochrane Collaboration)

 The suggested timeline for a Cochrane review is: 

• Preparation of protocol 1 – 2 months
• Searches for published and unpublished studies 3-8 months
• Pilot test of eligibility criteria 2-3 months
• Inclusion assessments 3-8 months
• Pilot test of ‘Risk of bias’ assessment 3 months
• Validity assessments 3-10 months
• Pilot test of data collection 3 months
• Data collection 3-10 months
• Data entry 3-10 months
• Follow up of missing information 5-11 months
• Analysis 8-10 months
• Preparation of review report 1-11 months
• Keeping the review up-to-date 12 months

A team: A systematic review can't be done alone! You need to work with subject experts to clarify issues related to the topic; librarians to develop comprehensive search strategies and identify appropriate databases; reviewers to screen abstracts and read the full text; a statistician who can assist with data analysis; and a project leader to coordinate and write the final report.

A clearly defined question: Clarify the key question(s) of you systematic review and the rationale for each question. Use the PICO framework to identify key concepts of the question. Determine inclusion/exclusion criteria.  Build a good clinical question to facilitate searching for a precise answer.

Structure of a clinical question:

• Patient population or problem - What are the significant characteristics of the patient or the population?
• Intervention - What is the main intervention or treatment?
• Comparison - Are you considering another intervention as a comparison to the first?
• Outcome - What is the effect of the treatment/intervention? (Outcome factor) How will you measure it?

A written protocol: You need to write a protocol outlining the study methodology. The protocol should include the rationale for the systematic review, key questions broken into PICO components, inclusion/exclusion criteria, literature searches for published/unpublished literature, data abstraction/data management, assessment of methodological quality of individual studies, data synthesis, and grading the evidence for each key question.

Official Protocol Resources:

• Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
  Evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.  Focused on randomized trials,  PRISMA can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions. The PRISMA statement consists of a 27 item checklist of items deemed essential for transparent reporting of a systematic review. Download the PRISMA Checklist
• The Cochrane Collaboration
• The Campbell Collaboration
• AMSTAR stands for A MeaSurement Tool to Assess systematic Reviews: Download the AMSTAR Checklist

A registered protocol: After you write the protocol, you should register it with PROSPERO, an International Prospective Register of Systematic Reviews. Registration is free and open to anyone undertaking systematic reviews of the effects of interventions and strategies to prevent, diagnose, treat, and monitor health conditions, for which there is a health related outcome.

Comprehensive literature searches: First, identify systematic reviews that may address your key questions. Then, identify appropriate databases and conduct comprehensive and detailed literature searches that can be documented and duplicated. Your research strategy has to be reproducible. 

Citation management: You should have working knowledge of EndNote to help manage citations retrieved from literature searches.

Follow reporting guidelines: Use appropriate guidelines for reporting your review for publication.

For detailed literature information about the process: See Handbooks for Doing Systematic Reviews

These are the steps of a Systematic Review adapted from Cochrane Canada 2011, Session three: A 'snapshot' of the steps of conducting a Cochrane Review (part 1).:

Steps of a review (PIECES)

• P: Planning-  the methods of the systematic review are generally decided before conducting it.  
• I: Identifying- searching for studies which match the preset criteria in a systematic manner.
• E: Evaluating- sort all retrieved articles (included or  excluded) and assess the risk of bias for each included study.
• C: Collecting/combining: each study is coded with preset form, either qualitatively or quantitatively synthesize data.
• E: Explaining:  placed results of synthesis into context, strengths and weaknesses of the studies. 
• S: Summarizing: report is provides description of methods and results in a clear and transparent manner.                                       

• Systematic Reviews: The Process   Guide from the Duke University Medical Center Library with a description of the systematic review process links to guidance and resources for conducting a systematic review.
• AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews The AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews, provides detailed information on selecting eligibility criteria, searching the literature, data extraction, quality assessment, and other steps in the creation of a systematic review.
• PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses.
• IOM Systematic Review Standards A list of standards for systematic reviews from the Institute of Medicine, including a list of the steps that a systematic review should include and what those steps should entail.
• Systematic Review: Q&A  A resource for finding data sources, filters, and standards to support systematic searches of the biomedical literature. (Harvard)

Library Liaisons can partner with you and advise on how to start a systematic review. 

Basic Service includes: 

• Provide overview of the systematic review (SR) process
• Inform researcher of SR standards (IOM, PRISMA, Cochrane)
• Assist researcher with formulating the research question
• Suggest databases for searching
• 1 meeting required

Students needing help with systematic reviews can contact liaison librarians to learn about the process.                               

Systematic review service will be  available to faculty, residents, fellows, and post-docs. 

Full Services includes: 

• Provide overview of the systematic review (SR) process
• Inform researcher of SR standards (IOM, PRISMA, Cochrane)
• Assist researcher with formulating the research question
• Suggest databases for searching
• Design the main search strategy
• Translate the search across multiple databases
• Conduct the search in multiple databases
• De-duplicate and merge the citations
• Deliver citations in preferred citation management software (EndNote, Refworks, etc)
• Provide basic guidance on screening process
• Provide researcher with several documents to include in publication (partially completed PRISMA flowchart, Citation Data Tracking, and search strategies)
• Assist in writing Methods section for publication
• Instruct how to obtain full text of articles through ILL if necessary
• Run update of searches prior to publication (as required by standards)
• At least 5 meetings required (once a month recommended)
• Librarian listed as contributing author is required for this option