Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How May U.S. Courts Scrutinize Their Recidivism Risk Assessment Tools? Contextualizing AI Fairness Criteria on a Judicial Scrutiny-based Framework

Tin Nguyen, Jiannan Xu, Phuong-Anh Nguyen-Le, Jonathan Lazar, Donald Braman, Hal Daumé, Zubin Jelveh

TL;DR

This work examines how recidivism risk assessment (RRA) tools intersect with fairness concepts from AI and law. It maps technical notions of procedural, group, and individual fairness onto U.S. constitutional guarantees (Due Process and Equal Protection) and identifies challenges in codifying these ideas into statutes and regulations. The authors propose a three-threshold, scrutiny-based framework that treats demographic features as continuous scrutiny ranges to guide which inputs and fairness criteria auditors should emphasize. They further situate their US-centric framework in EU, China, and India to illustrate cross-jurisdictional differences and discuss implications for tool designers, users, and decision subjects in shaping fair and accountable AI in high-stakes contexts.

Abstract

The AI/HCI and legal communities have developed largely independent conceptualizations of fairness. This conceptual difference hinders the potential incorporation of technical fairness criteria (e.g., procedural, group, and individual fairness) into sustainable policies and designs, particularly for high-stakes applications like recidivism risk assessment. To foster common ground, we conduct legal research to identify if and how technical AI conceptualizations of fairness surface in primary legal sources. We find that while major technical fairness criteria can be linked to constitutional mandates such as ``Due Process'' and ``Equal Protection'' thanks to judicial interpretation, several challenges arise when operationalizing them into concrete statutes/regulations. These policies often adopt procedural and group fairness but ignore the major technical criterion of individual fairness. Regarding procedural fairness, judicial ``scrutiny'' categories are relevant but may not fully capture how courts scrutinize the use of demographic features in potentially discriminatory government tools like RRA. Furthermore, some policies contradict each other on whether to apply procedural fairness to certain demographic features. Thus, we propose a new framework, integrating U.S. demographics-related legal scrutiny concepts and technical fairness criteria, and contextualize it in three other major AI-adopting jurisdictions (EU, China, and India).

How May U.S. Courts Scrutinize Their Recidivism Risk Assessment Tools? Contextualizing AI Fairness Criteria on a Judicial Scrutiny-based Framework

TL;DR

This work examines how recidivism risk assessment (RRA) tools intersect with fairness concepts from AI and law. It maps technical notions of procedural, group, and individual fairness onto U.S. constitutional guarantees (Due Process and Equal Protection) and identifies challenges in codifying these ideas into statutes and regulations. The authors propose a three-threshold, scrutiny-based framework that treats demographic features as continuous scrutiny ranges to guide which inputs and fairness criteria auditors should emphasize. They further situate their US-centric framework in EU, China, and India to illustrate cross-jurisdictional differences and discuss implications for tool designers, users, and decision subjects in shaping fair and accountable AI in high-stakes contexts.

Abstract

The AI/HCI and legal communities have developed largely independent conceptualizations of fairness. This conceptual difference hinders the potential incorporation of technical fairness criteria (e.g., procedural, group, and individual fairness) into sustainable policies and designs, particularly for high-stakes applications like recidivism risk assessment. To foster common ground, we conduct legal research to identify if and how technical AI conceptualizations of fairness surface in primary legal sources. We find that while major technical fairness criteria can be linked to constitutional mandates such as ``Due Process'' and ``Equal Protection'' thanks to judicial interpretation, several challenges arise when operationalizing them into concrete statutes/regulations. These policies often adopt procedural and group fairness but ignore the major technical criterion of individual fairness. Regarding procedural fairness, judicial ``scrutiny'' categories are relevant but may not fully capture how courts scrutinize the use of demographic features in potentially discriminatory government tools like RRA. Furthermore, some policies contradict each other on whether to apply procedural fairness to certain demographic features. Thus, we propose a new framework, integrating U.S. demographics-related legal scrutiny concepts and technical fairness criteria, and contextualize it in three other major AI-adopting jurisdictions (EU, China, and India).
Paper Structure (24 sections, 4 figures)

This paper contains 24 sections, 4 figures.

Table of Contents

  1. Introduction
  2. Background: Conceptualization of Fairness
  3. HCI vs. AI
  4. HCI/AI vs. Law
  5. Methodology -- Legal Research
  6. Legal Research in HCI
  7. How?
  8. Interaction among U.S. sources of law
  9. Procedural Fairness and Outcome (Group and Individual) Fairness Are All Constitutionally Relevant
  10. Specific Policies for RRA Touch On Procedural and Group Fairness, but not Individual Fairness
  11. Demographic Features in the Same Judicial "Scrutiny" Category Might Trigger Slightly Different Scrutiny by U.S. Courts in Practice
  12. Jurisdictions Disagree on Procedural Fairness Criteria (Which Demographics to Use as Model Inputs)
  13. Proposal: Integrating Fairness Criteria and Demographic Features into a Scrutiny-based Framework
  14. International Perspectives: How our U.S. Law Findings Relate to Other Major Jurisdictions
  15. The European Union
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Whether Individual, Group, and Procedural Fairness Concepts are recognized by the U.S. Constitution, case law (judicial), statutes (legislative), or regulations (executive)
  • Figure 2: Potential ordering of proposed scrutiny thresholds.
  • Figure 3: Legal sources classify demographic features into three scrutiny ranges. For RRA, only strict scrutiny features are above the "exclusion from model inputs" (yellow) scrutiny threshold, i.e., expected by legal sources to be excluded from the AI models' input space. There are conflicting legal sources about whether intermediate scrutiny or rational basis features (such as sex/gender or age) should be excluded.
  • Figure 4: How the empirical findings by nguyen2025demographic might be applied to our scrutiny-based framework.