Systematic reviews, scoping reviews and other evidence syntheses

Search strategy

• refers to the specialized combination of keywords, subject headings, search syntax and logic operators used to retrieve the optimal number of studies for an evidence synthesis project
• your search strategy should be comprehensive and designed to collect all available relevant evidence on a research question, including peer-reviewed and grey literature; ideally, unpublished and non-English sources should be included to reduce bias
• multiple sources must be searched because:
  • content varies across different subject databases - no one database includes all relevant content
  • Not all research is published in journals, or published at all

Reporting search strategies

Search strategies are critical components of evidence synthesis research and are:

• documented and described in the Methods section of published reviews

• included as an Appendix in published reviews

Follow guidance from PRISMA on how to report search strategies in a systematic review:

Rethlefsen, M.L., Kirtley, S., Waffenschmidt, S. et al. PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews. Syst Rev 10, 39 (2021). https://doi.org/10.1186/s13643-020-01542-z

Expert guidance from librarians

Key evidence synthesis organizations (IOM, Cochrane, CIHR, Campbell Collaboration) highlight the importance of librarian expertise in developing search strategies which are:

• Free from bias as possible
• Transparent 
• Reproducible

Building a search strategy

Expect an iterative process, beginning with preliminary searching before developing a final comprehensive search strategy.

• Preliminary searching:
  • Identify key terms, synonyms and subject headings to use in your search
  • Find existing systematic reviews on any component of your topic and review search strategies included in the methods or appendix
• Comprehensive searching:
  • Develop a full search strategy including:
    • keywords, synonyms and subject headings
    • appropriate search syntax and logic operators
  • Identify appropriate subject databases for your topic (e.g. MEDLINE, CINAHL)
    • important to search in multiple databases because each database has unique content; evidence syntheses searching must be as comprehensive as possible
  • Run a trial search in one database to test its efficacy; revise search as needed
  • Run searches in other relevant databases

Other sources to search

• Determine relevant grey literature sources and search with modified strategies
• Hand search additional relevant non-indexed journals or conference proceedings, if appropriate
• Contact experts, including corresponding authors of included studies, to find out about additional or ongoing studies

Document your searches

• Best practices for evidence synthesis research require transparency and reproducibility: documentation supports this
• Searching is iterative and requires a significant investment of time: you will need to refer back to your search decisions months later and may need to respond to reviewers' questions about those decisions
• Reporting standards require you to report on search strategies in the methods section of your review project

• Use this spreadsheet:

You will develop a sophisticated combination of keywords, subject headings, search syntax and logic operators to retrieve the optimal number of studies for an evidence synthesis project

• Keywords -- natural language words describing the main concepts in your research e.g. if your research question is investigating the effects of social media on adolescents' well-being, your topic keywords are:
  • social media
  • well-being
  • adolescents 
    • Always generate synonyms and related terms for each keyword to ensure you capture studies which investigate the same topic but use different descriptors e.g. adolescent - teenager, young adult, youth, etc.
• Subject headings -- controlled vocabulary assigned by databases to articles about specific topics; helps broaden your search by ensuring you retrieve relevant studies which may use varying keywords

Search both keywords and subject headings

A search strategy must include both keywords and subject headings to be comprehensive and avoid these pitfalls:

• natural language requires using multiple synonyms and variations
• articles may not be correctly indexed
• there may be no subject heading for your concept

Search syntax and techniques

• Truncation -- use an asterisk (or other symbol specific to a database) to retrieve alternate word endings; broadens your search. e.g. teen* retrieves teens, teenager, teenagers, teenaged
• Exact phrase -- enclose 2-3 words or a short phrase in quotes to ensure they are searched together instead of independently e.g. "social media"
• Logic operators -- connect your search terms to both broaden and focus your search results
  • OR -- broadens search by retrieving ANY of the search terms e.g. teen* OR adolescent OR "young adult"
  • AND -- focuses search by retrieving ALL of the search terms e.g. teen* AND "social media"
  • NOT -- focuses search by removing irrelevant results e.g. "social media" NOT Twitter; use with care to avoid missing studies that include both of your terms
• Adjacency searching -- use a proximity operator e.g. NEAR, WITHIN to find words which appear within a certain number of words of each other e.g. social N3 media; social W2 media
• Parentheses -- connect related words e.g. ("social media" OR Twitter OR Facebook) AND (adolescent* OR "young adult*")

Search tutorials and tips

Documentation and tutorials for core and common subject resources

Systematic review database searching cheat sheet - health focus

Subject and niche databases

Translating searches to multiple databases

Search limits

• Database search results can often be limited by e.g. year, gender, species, etc.
• Generally, avoid using these filters as they introduce bias into your search results
• Database search limiters are appropriate when:
  • updating an existing search
  • updating an existing systematic review

Search filters

• Also called hedges, these are pre-designed and validated search strings which allow you to filter your search results e.g. by study design without sacrificing sensitivity/specificity or introducing major sources of bias
• Always use a critically appraised search filter from one of these organizations:
  • ISSG Search Filters Resource
    • search filters for study designs and topics such as adverse effects and quality of life
    • guidance on how to critically evaluate filters
      
  • McMaster University Search Filters
    • filters (called hedges) for therapy, diagnosis, prognosis, causation, economics and more
  • SIGN Search Filters
    • filters for systematic reviews, randomized controlled trials, observational studies, diagnostic studies, economic studies, patient issues
  • Strings Attached: CADTH's database search filters
    • filters for systematic reviews, meta-analysis, randomized controlled trials, guidelines and more