Efficiency vs. Equity: A Structured Interdisciplinary Review of AI in Criminal Justice Risk Assessments

Risk assessment tools are used in criminal justice to evaluate an individual’s likelihood of reoffending. There is a growing discussion around the use of artificial intelligence (AI) and machine learning (ML) in algorithmic risk assessment (ARA). This survey examines the use of and the potential for bias in the use of ARA in criminal justice. Through a structured interdisciplinary review of recent research on the impact of ARA, this investigation examines the tools currently being used and whether there is evidence that ARA tools contribute to bias. Included papers were collected from Google Scholar and the ACM Digital Library and have been published since 2015, discuss AI, and focus on the adult justice system in the US, yielding 56 studies. In total, 79% of the surveyed literature concluded that AI and ML can or do contribute to biased performance in risk assessment. The two most recorded sources of bias were the use of historical court records as training data and the use of variables or features that correlate strongly with race, gender, age, or other protected attributes, while noting that this result relies heavily on a small number of real-world observations, most notably the COMPAS dataset collected in Broward County. The recorded benefits of ARA included efficiency and resource utilization. The use of AI-derived risk assessment tools is growing and holds the potential to affect a lot of lives. It is important to examine and consider the implications of their use, especially involving bias and fairness in criminal justice decision-making.