Library Help Chat Skip to Main Content

Evidence Synthesis

A guide to evidence synthesis methods such as systematic reviews, scoping reviews, and structured literature reviews.

Meta Analysis in Criminal Justice

Turanovic, J. J., & Pratt, T. C. (2020). Meta-Analysis in Criminology and Criminal Justice: Challenging the Paradigm and Charting a New Path Forward. Justice Evaluation Journal, 4(1), 21–47. https://doi.org/10.1080/24751979.2020.1775107

Abstract: Meta-analyses are appearing more frequently in the criminological literature. Yet the methods typically used are guided by a methodological paradigm that risks producing meta-analyses of limited value. Here we outline three key methodological issues that meta-analysts face and we present a methodological challenge to the dominant meta-analysis paradigm. We focus specifically on: (1) inclusion criteria, (2) analysis of bivariate versus multivariate effect sizes, and (3) methods for handling statistical dependence. Issues of reproducibility and recommendations for moving forward are discussed.

Pratt, T. C. (2010). Meta‐Analysis in Criminal Justice and Criminology: What It is, When It’s Useful, and What to Watch Out for. Journal of Criminal Justice Education, 21(2), 152–168. https://doi.org/10.1080/10511251003693678

Abstract: Meta‐analyses are becoming more common in the criminal justice and criminological literature. While much of the published work related to the technique is focused on the more technical aspects of the statistical methods that have been developed for meta‐analysis, less common are broader—and perhaps more critical—discussions concerning various issues associated with the method. Accordingly, this article presents an overview of meta‐analysis in the context of what have become three of the more important issues within the meta‐analysis literature in recent years: (1) the conditions under which meta‐analyses are, and are not, most useful, (2) the dilemma of whether or not to include unpublished work in the sample of studies to be analyzed, and (3) the choice of bivariate versus multivariate effect size estimates to be synthesized. The objective is to take these issues out of what has arguably been debates about technical orthodoxy and instead to place them into a broader research context within criminal justice and criminology.

Kim, B. (2022). Publication bias: A “bird’s-eye view” of meta-analytic practice in criminology and criminal justice. Journal of Criminal Justice, 78, 101878. https://doi.org/10.1016/j.jcrimjus.2021.101878

Abstract: Recent CCJ meta-analytic studies have at least partially addressed the issue of publication bias by employing systematic search and statistical methods. However, the current state of CCJ meta-analyses does not meet the expectation required in medical science and psychology that all meta-analytic reviews report the range of effect size estimates across multiple publication bias detection and correction tests. The statistical methods commonly used for assessing publication bias are applied without testing and interpreting assumptions about the missing studies.

There is a need to continue monitoring the quality of meta-analyses to gain a comprehensive picture of how bias leaves a potential imprint in CCJ research.
 

Copyright | Accessibility | Terms of Use