Evidence Synthesis

A guide to evidence synthesis methods such as systematic reviews, scoping reviews, and structured literature reviews.

Scoping Reviews

Scoping Reviews aim to provide a preliminary assessment a body of literature and chart, or map, the the conceptual boundaries of a topic or field. They may give an overview of or clarify contested concepts in a field, identify and analyze gaps, or identify the nature and extent of research evidence on a topic. They may be done as a precurser to a systematic review, or in areas that are too 'messy' for a systematic review. They may result in a concept or policy map.

Steps to a Student Scoping Review

Note that a scoping review is not necessarily "smaller" than a Systematic Review - they can often be larger projects, if they are more broadly defined.

1) Formulate a solid research area

It will be much easier for you to perform a structured information search if you first define and delimit your research question or topic in a clear way. A scoping review topic may be broad, and not formulated as concretely as a research question, but still needs to be bounded and specific enough to be achievable. For students, this means making sure the question (or topic) is researchable - that there is a body of evidence already collected that can speak to the subject.
One way to define and structure your topic is to break it down into different parts. Have a clear objective, and one or more questions aligned with it.
The more specific and focused your research topic is, the easier the research will be. Consider using PCC - Population, Concept, Context - or PICO - Population, Intervention, Comparison, Outcome - to think through and narrow down your research topic. These are research question frameworks commonly used for scoping reviews that can be used for breaking down a research question into different parts.
You also need to define the most important key concepts of your research question.
If you are following the PRISMA checklist, this will contribute to your structured summary, rationale, and objectives. (See the tip sheets for more detail).

It is highly likely that, as you start to research the area, you will have to modify your topic and concepts. Keep track of any changes you make, and why.

2) Search Strategy - Find Search Terms and Make Search Concept Blocks

Some general search tips can be found on the Search tab of this guide. For a scoping review, the goal is to have a transparent and replicable search process. You are aiming to identify all of the results that are relevant to your topic (which is why defining the topic in step one is important). This requires designing searches that are well balanced in terms of precision and sensitivity. To achieve this you will need to design sophisticated searches that leverage controlled vocabularies and search syntaxes.

A good way of achieving a structured final search query is to arrange your search terms into search blocks.
Start with your concepts, key words, and PCC/PICO terms from step 1. Brainstorm synonyms, related concepts, and antonyms. Document these concept clusters/ search blocks as you go!
Test searching is a good way to investigate the terminology of a subject area and find search terms, as well as to know if there is research that has been done on a topic.
Reading key articles can help you gather additional search terms for your final search strategy.
Explore your subject databases, and pay special attention to the keywords and subject headings used for relevant articles.
Find free-text search terms by investigating what words that occur in the title and abstract of relevant articles.
Remember to search, and acknowledge, outdated and offensive terminology if it is relevant to your topic.
Use nesting, Boolean and wildcards to improve and refine your search blocks.
Your search terms will be refined as you search, but you will need to get to a "final" series of searches/ search strings, which are the systematic part of a scoping review.
Searching is always about finding the balance between precision (getting exact results with fewer unrelated results to filter out, but likely missing some relevant results) and specificity (casting a wide net and likely catching most relevant results, but having to filter out more relevant results).

3) Search in a Structured Way and Document Your Searches

To get a comprehensive search result, you will need to search for articles in several different databases. For a scoping review, you likely will need to consider grey literature sources, such as theses or reports. Some journals may not be indexed in databases, and may need to be searched independently.
Your search strategy should be as uniform as possible in every database, but you may have to adapt your use of subject headings.
It can be very useful to find other systematic reviews or evidence synthesis that may have search blocks you can reuse.
Make sure you are documenting your search queries, including date searched, amount and relevancy of results, as well as any modifications you have made. (documentation template)
There is no abstract ideal number of results for scoping review searches - but if you are getting too many or too few to complete the assignment, you may need to reevaluate your question or topic. Note that this is not considered part of the process for a "pure" scoping review, but may be part of the preliminary work before a protocol is registered.

Refining your strategy:

Are all your key articles present in the search results, or are there some key articles that your search query is unable to retrieve?
Are you getting too few search hits? Investigate why. Perhaps you need to remove one of your search blocks, add one or several synonyms within a search block, or search for parts of words by truncating one or several of your free-text search terms, in order to broaden your search?
Does your search strategy generate too many non-relevant search hits that have nothing to do with your research question? Investigate why. Perhaps you need to add another search block, remove one of the synonyms from one of your search blocks, or search for phrases by enclosing one or several of your free-text terms in quotation marks, in order to narrow your search?
More tips on how to improve your search strategy.

From: Structured literature reviews – A guide for students CC-BY-SA 4.0

4) Determine Inclusion and Exclusion Criteria

Inclusion and exclusion criteria help create a replicable way to decide which articles are included or excluded from your review. They may include some of the key concepts, such as date of the study, population or location, that haven't been faceted out in your search. In a true Scoping Review, these criteria are a priori, and should be defined prior to search strategy, to reduce bias. They should be noted as part of the methodology.

See: https://libguides.jcu.edu.au/scoping/criteria and https://guides.library.utoronto.ca/c.php?g=713309&p=5088425

5) Develop a Protocol

A protocol documents your topic, rationale, hypothesis, and intended methodology. Most scoping review instructions/ checklists will include this step earlier in the process, and require that the protocol is made publicly available before the study runs. However, this may not be feasible for students.

Your protocol may include:

Team members (to ensure replicability and reduce bias)
Objective of the scoping review
Inclusion/ exclusion criteria
Key details of the search strategy – databases, search terms
Processes and tools for screening and quality assessment
Data analysis methods that may be used

Example of a JBI Protocol Form for Scoping Reviews (will download an editable Word doc).

Adapted from: https://guides.lib.unc.edu/scoping-reviews/protocol

6) Locate Studies, Manage Citations

Now you get to run your searches, collect your articles, and screen which ones will be included in the study or not, based on your inclusion/ exclusion criteria.

To keep track of the articles that result from searches, take advantage of tools built into most library databases, such as exporting the bibliographic information from search results.

UWinnipeg Library does not currently have a subscription to Covidence, which is a tool frequently used to manage systematized search results between teams. We suggest using Zotero to manage citations, but there are limitations, as it automatically ‘dedupes’ results.

For more on citation management options, and how to import/ export references: https://guides.lib.unc.edu/scoping-reviews/manage-citations

7) Screen & Review Studies

Most scoping studies will involve an initial screening stage, where results are quickly eliminated by scanning the title and abstract, then a second screening stage where the ‘short list’ is compared to the inclusion/ exclusion criteria. See: https://guides.library.unisa.edu.au/ScopingReviews/HowToScreen

UWinnipeg Library does not have subscriptions for screening tools. You can use Excel to track screening decisions. See: Video: Rob Morissette and https://www.yopl.info/post/excel-workbooks-and-user-guides-for-systematic-reviews

8) Analyze and Synthesize – write it up!

- Scoping Reviews include original thought in how the analysis and synthesis of data is presented. You may use frameworks, matrices, maps, or other ways that are relevant to your discipline and topic. As well as synthesizing and analyzing the evidence, you should provide some discussion, which may include limitations and/ or conclusions.

- For guidance on data extraction, see: https://guides.library.utoronto.ca/c.php?g=713309&p=5104947 and https://guides.lib.unc.edu/scoping-reviews/extract-data and https://journals.lww.com/jbisrir/fulltext/2023/03000/recommendations_for_the_extraction,_analysis,_and.7.aspx

- include search decision flowcharts or tables, showing publications found and reviewed, and how the inclusion/ exclusion criteria were applied

- you can follow a checklist format, such as PRISMA or JBI – see: https://guides.library.unisa.edu.au/ScopingReviews/ReportAndWrite

- writing a Scoping Review should always conform to academic citation standards