Evidence Without Injustice: A New Counterfactual Test for Fair Algorithms

TL;DR

The paper argues that traditional fairness criteria fail to capture how structural injustice shapes the evidential value of algorithmic outputs. It introduces the Counterfactual Independence Principle (CIP), which asks whether an evidentiary relation $E$ to an outcome $O$ would survive in nearby worlds lacking unjust social mechanisms, and uses this to distinguish punitive versus supportive uses of evidence. Applying CIP to policing shows that predictive policing based on historical data relies on unjust structures and thus passes neither moral nor CIP scrutiny for punitive use, whereas a camera-based, diagnostic approach remains probative in just worlds and thus is morally preferable. The CIP framework is then extended to healthcare and broader policy, arguing that evidence tainted by structural injustice should guide remedial, supportive actions rather than punitive measures, with careful consideration of downstream interventions. Overall, CIP reframes fairness by focusing on the epistemic resilience of evidence to structural injustice and provides a principled guide for downstream use across domains.

Abstract

The growing philosophical literature on algorithmic fairness has examined statistical criteria such as equalized odds and calibration, causal and counterfactual approaches, and the role of structural and compounding injustices. Yet an important dimension has been overlooked: whether the evidential value of an algorithmic output itself depends on structural injustice. We contrast a predictive policing algorithm, which relies on historical crime data, with a camera-based system that records ongoing offenses, where both are designed to guide police deployment. In evaluating the moral acceptability of acting on a piece of evidence, we must ask not only whether the evidence is probative in the actual world, but also whether it would remain probative in nearby worlds without the relevant injustices. The predictive policing algorithm fails this test, but the camera-based system passes it. When evidence fails the test, it is morally problematic to use it punitively, more so than evidence that passes the test.