Forecasting Crime, Forecasting Police: A Critical Review of Predictive Policing and Algorithmic Risk Assessment in American Criminal Justice

Abstract

Two decades have passed since the first place-based forecasting pilots, and we could not develop a fair system of algorithmic forecasting in American policing. The evidentiary record remains thin and contested. This is affecting every stage of the American justice system. The existing systems rely on historical data which consists of enforcement and not of offending. This is important when decisions are made to allocate resources to patrolling, pretrial releases, sentencing and awarding paroles to prisoners. This review considers the problems identified in present practices. These problems consist of using enforcement data instead of offending data, feedback loops in the forecasting models, disproportionate burden on minorities like Black, Latino and poor communities, and weak accountability. Feedback loops happen when the data generated by the biased datasets is automatically used by the systems to train themselves, which increases the bias. Accountability remains weak because the process is not transparent and the bias remains hidden in the core dataset used to initially train the model. This review also discusses literature on predictive risk assessment in courts and prisons, where the COMPAS controversy exposed competing definitions of fairness. Across both literatures the evidence of benefit is modest and uneven while the evidence of harm is credible. In the end this review closes with the National Institute of Justice's current emphasis on the evaluation of the use of AI in courts, policing and prisons. This review suggests more focus on the outcomes of statistical predictions instead of evaluating the accuracy of statistical predictions.