Detecting Statistically Significant Fairness Violations in Recidivism Forecasting Algorithms

TL;DR

The paper addresses the challenge of determining whether fairness violations in recidivism forecasting are statistically significant rather than due to chance. It develops a rigorous framework that uses k-fold cross-validation to generate sampling distributions for a broad set of fairness metrics and applies formal statistical tests across multiple bias definitions. Through an empirical study on National Institute of Justice data with four algorithms, the study finds widespread statistically significant bias against Black individuals under several metrics, and in some definitions observes reverse discrimination against White individuals. The work highlights the importance of robust statistical testing and cross-definition evaluation when assessing algorithmic fairness in high-stakes criminal justice applications.

Abstract

Machine learning algorithms are increasingly deployed in critical domains such as finance, healthcare, and criminal justice [1]. The increasing popularity of algorithmic decision-making has stimulated interest in algorithmic fairness within the academic community. Researchers have introduced various fairness definitions that quantify disparities between privileged and protected groups, use causal inference to determine the impact of race on model predictions, and that test calibration of probability predictions from the model. Existing literature does not provide a way in which to assess whether observed disparities between groups are statistically significant or merely due to chance. This paper introduces a rigorous framework for testing the statistical significance of fairness violations by leveraging k-fold cross-validation [2] to generate sampling distributions of fairness metrics. This paper introduces statistical tests that can be used to identify statistically significant violations of fairness metrics based on disparities between predicted and actual outcomes, model calibration, and causal inference techniques [1]. We demonstrate this approach by testing recidivism forecasting algorithms trained on data from the National Institute of Justice. Our findings reveal that machine learning algorithms used for recidivism forecasting exhibit statistically significant bias against Black individuals under several fairness definitions, while also exhibiting no bias or bias against White individuals under other definitions. The results from this paper underscore the importance of rigorous and robust statistical testing while evaluating algorithmic decision-making systems.