Do structured risk assessments predict violent, any, and sexual offending better than unstructured judgment? An umbrella review.

Although it is widely believed that risk assessment tools lead to more accurate estimates of risk of violence and offending than unstructured clinical judgments, the nature and quality of evidence that supports this view is unclear. As such, we conducted an umbrella review of systematic reviews. Through a search of 15 databases, we identified nine systematic reviews, including six meta-analyses and three narrative systematic reviews, that compared unstructured and structured risk judgments for any, violent, and sexual offending. Each review was independently coded by two raters. Raters also coded the 46 primary studies on unstructured judgment included in these reviews. Although the reviews concluded that structured risk judgments are superior to unstructured judgments, the data supporting these conclusions have limitations. None of the systematic reviews directly compared risk assessment tools with unstructured judgments. In addition, two thirds of the primary studies included in the systematic reviews were from the 1980s or earlier, and 89% had serious methodological limitations that created a high risk of bias. In many cases, the primary studies did not examine unstructured judgments per se but instead used proxies such as legal and administrative decisions. As such, there is a pressing need for an updated systematic review that focuses on direct comparison studies and carefully addresses study limitations. To address this gap, we have initiated a preregistered systematic review.