A Comparative Simulation Study of the Fairness and Accuracy of Predictive Policing Systems in Baltimore City

TL;DR

This paper addresses whether predictive policing (PredPol) and hot spots policing (KDE variants) produce fairer and more accurate outcomes in Baltimore using a 300-day agent-based simulation. It compares these approaches across eight settings with varying officer counts and crime data (TOTAL vs AGG. ASSAULT), measuring racial fairness gaps and neighborhood-level disparities via metrics such as $\text{RacialFairnessGap}_{\text{PoliceShare}} = |\overline{P_{Black}} - \overline{P_{White}}|$, $\text{PCR}_i = \frac{P_i}{C_i}$, Gini coefficients, and CoverageAccuracy $= \frac{\text{Total Detected Crimes}}{\text{Total Crimes}}$. The study finds that PredPol is generally more accurate and fairer in the short term, but it exhibits faster bias amplification, whereas long-term KDE can achieve comparable accuracy at the cost of worse racial fairness, and that Baltimore-specific data sometimes shift bias toward White neighborhoods when all crime records are used. The results demonstrate the necessity of city-specific, pre-deployment evaluation frameworks to identify long-term inequities and to guide responsible deployment of policing technologies. {The metrics and framework rely on explicit definitions and expressions, such as $PCR_i$ and $Gini$, to quantify resource distribution and inequality across neighborhoods.}

Abstract

There are ongoing discussions about predictive policing systems, such as those deployed in Los Angeles, California and Baltimore, Maryland, being unfair, for example, by exhibiting racial bias. Studies found that unfairness may be due to feedback loops and being trained on historically biased recorded data. However, comparative studies on predictive policing systems are few and are not sufficiently comprehensive. In this work, we perform a comprehensive comparative simulation study on the fairness and accuracy of predictive policing technologies in Baltimore. Our results suggest that the situation around bias in predictive policing is more complex than was previously assumed. While predictive policing exhibited bias due to feedback loops as was previously reported, we found that the traditional alternative, hot spots policing, had similar issues. Predictive policing was found to be more fair and accurate than hot spots policing in the short term, although it amplified bias faster, suggesting the potential for worse long-run behavior. In Baltimore, in some cases the bias in these systems tended toward over-policing in White neighborhoods, unlike in previous studies. Overall, this work demonstrates a methodology for city-specific evaluation and behavioral-tendency comparison of predictive policing systems, showing how such simulations can reveal inequities and long-term tendencies.

Figures (1)

• Figure 2: Simulation maps comparing three algorithms (columns). The top row shows police distribution on Day 1, while the bottom row shows distribution on day 300. All were simulated using report-probability $0.4$ and $40$ officers.