Skip to main content



Systematic reviews, scoping reviews and other evidence syntheses

Overview of evidence synthesis reviews and relevant strategies, tools and resources.

Data extraction

  • After full-text screening, the step step is extracting data to:
    • Objectively and accurately summarize studies in a common format to facilitate synthesis,
    • Identify numerical data if a meta-analysis is to take place, and
    • Obtain information to objectively assess the risk of bias in, and applicability of, studies.
  • All reviewers must use the same, standardized form to extract and record data:
"The data collection form is a bridge between what is reported by the original investigators (e.g. in journal articles, abstracts, personal correspondence) and what is ultimately reported by the review authors." -- Rationale for data collection forms, Cochrane Handbook for Systematic Reviews of Interventions
  • A fair procedure for resolving discrepancies should be established

Data extraction tools

Data can be extracted and recorded using:

What information to record

  • Use the review's research question and inclusion and exclusion criteria as a guide
  • Elements recorded will vary according to your project, but standard elements include:
  • full article citation
  • study objectives
  • study design
  • participant characteristics
  • intervention
  • comparison
  • results/outcome
  • comments related to study quality

Study characteristics table

  • Use data extraction forms to produce a summary table of study characteristics
  • Include the summary table in the Results section of the report of the systematic review, either as an appendix or in the text

Quality appraisal

Critical appraisal is a key component of many evidence syntheses reviews.

"Critical appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context" (Burls, 2009). 

Generally, the critical appraisal process will consider:

  • Question - Does this study address a clearly focused question?
  • Methodological quality - Did the study use valid methods to address this question?  To what extent do the study design and conduct eliminate the potential for systematic error (bias)?
  • Precision – What is the likelihood of random errors? (Often depicted as the confidence interval around the result)
  • External validity - Are these valid, important results applicable to my patient or population?

To determine if your evidence synthesis review requires critical appraisal, consult this chart which outlines 14 types of review articles and indicates whether quality appraisal is typically included. Table 1: Main review types characterized by methods used (Grant & Booth, 2009).

Critical appraisal tools

When reading the full text of each article identified for inclusion in the review as part of the data extraction process, apply a Critical Appraisal Tool to each study selected for inclusion. Select a critical appraisal tool appropriate for the study designs you are reviewing.

Appraising systematic reviews
 

References

Systematic Reviews in Health by Murray Turner at University of Canberra

Systematic Reviews by Lydia Thorne at University of Ontario Institute of Technology

Systematic reviews and other syntheses by Sandra McKeown, Queen's University Library

Burls, A. (2009). What is critical appraisal? Hayward Medical Communications.

Grant, M. J., & Booth, A. (2009). A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26(2), 91-108.