Evidence Synthesis

A guide to evidence synthesis methods such as systematic reviews, scoping reviews, and structured literature reviews.

Mixed Methods

Johnson, S. D., Tilley, N., & Bowers, K. J. (2015). Introducing EMMIE: An evidence rating scale to encourage mixed-method crime prevention synthesis reviews. Journal of Experimental Criminology, 11(3), 459–473. https://doi.org/10.1007/s11292-015-9238-7

Abstract:

Objectives

This paper describes the need for, and the development of, a coding system to distil the quality and coverage of systematic reviews of the evidence relating to crime prevention interventions. The starting point for the coding system concerns the evidence needs of policymakers and practitioners.

Methods

The proposed coding scheme (EMMIE) builds on previous scales that have been developed to assess the probity, coverage and utility of evidence both in health and criminal justice. It also draws on the principles of realist synthesis and review.

Results

The proposed EMMIE scale identifies five dimensions to which systematic reviews intended to inform crime prevention should speak. These are the Effect of intervention, the identification of the causal Mechanism(s) through which interventions are intended to work, the factors that Moderate their impact, the articulation of practical Implementation issues, and the Economic costs of intervention.

Conclusions

Systematic reviews of crime prevention, and the primary studies on which they are based, typically address the question of effect size, but are often silent on the other dimensions of EMMIE. This lacuna of knowledge is unhelpful to practitioners who want to know more than what might work to reduce crime. The EMMIE framework is intended to encourage the collection of primary data regarding these issues and the synthesis of such knowledge in future systematic reviews.

Qualitative Evidence Synthesis

Flemming, K., & Noyes, J. (2021). Qualitative Evidence Synthesis: Where Are We at? International Journal of Qualitative Methods, 20. https://doi.org/10.1177/1609406921993276

Abstract

Qualitative evidence syntheses (QES) have increased in prominence and profile over the last decade as a discrete set of methodologies to undertake systematic reviews of primary qualitative research in health and social care and in education. The findings from a qualitative evidence synthesis can enable a richer interpretation of a particular phenomenon, set of circumstances, or experiences than single primary qualitative research studies can achieve. Qualitative evidence synthesis methods were developed in response to an increasing demand from health and social professionals, policy makers, guideline developers and educationalists for review evidence that goes beyond “what works” afforded by systematic reviews of effectiveness. The increasing interest in the synthesis of qualitative research has led to methodological developments documented across a plethora of texts and journal articles. This “State of the Method” paper aims to bring together these methodological developments in one place, contextualizing advances in methods with exemplars to support readers in making choices in approach to a synthesis and aid understanding. The paper clarifies what a “qualitative evidence synthesis” is and explores its role, purpose and development. It details the kind of questions a QES can explore, the processes associated with a QES, including the methods for synthesis. The rational and methods for integrating a QES with systematic reviews of effectiveness are also detailed. Finally approaches reporting and recognition of what a “good” or rigorous QES look like are provided.