Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical Guarantees in the Search for Less Discriminatory Algorithms

Chris Hays, Ben Laufer, Solon Barocas, Manish Raghavan

TL;DR

The paper addresses certifying sufficiency in searching for less discriminatory algorithms by casting model retraining as an optimal stopping problem and introducing an adaptive stopping algorithm that delivers high-probability upper bounds on the gains from further search, enabling evidence that the search is sufficiently exhaustive. It decomposes the marginal gain into conditional expected improvement and improvement probability, providing bounds $\\bar{\\mu}$ and $\\bar{p}_t(\\delta)$ under various assumptions, and develops both full-information and data-driven, anytime-valid guarantees for stopping, even with finite data. Empirical validation on credit, employment, and housing datasets shows substantial model multiplicity in disparate impact and modest accuracy loss when minimizing disparity, with the method providing conservative stopping certificates in practice. The framework generalizes beyond LDAs to certify searches over hyperparameters or other model classes, offering a principled, regulator-friendly tool for documenting search adequacy and guiding resource allocation in high-stakes ML pipelines.

Abstract

Recent scholarship has argued that firms building data-driven decision systems in high-stakes domains like employment, credit, and housing should search for "less discriminatory algorithms" (LDAs) (Black et al., 2024). That is, for a given decision problem, firms considering deploying a model should make a good-faith effort to find equally performant models with lower disparate impact across social groups. Evidence from the literature on model multiplicity shows that randomness in training pipelines can lead to multiple models with the same performance, but meaningful variations in disparate impact. This suggests that developers can find LDAs simply by randomly retraining models. Firms cannot continue retraining forever, though, which raises the question: What constitutes a good-faith effort? In this paper, we formalize LDA search via model multiplicity as an optimal stopping problem, where a model developer with limited information wants to produce strong evidence that they have sufficiently explored the space of models. Our primary contribution is an adaptive stopping algorithm that yields a high-probability upper bound on the gains achievable from a continued search, allowing the developer to certify (e.g., to a court) that their search was sufficient. We provide a framework under which developers can impose stronger assumptions about the distribution of models, yielding correspondingly stronger bounds. We validate the method on real-world credit, employment and housing datasets.

Statistical Guarantees in the Search for Less Discriminatory Algorithms

TL;DR

The paper addresses certifying sufficiency in searching for less discriminatory algorithms by casting model retraining as an optimal stopping problem and introducing an adaptive stopping algorithm that delivers high-probability upper bounds on the gains from further search, enabling evidence that the search is sufficiently exhaustive. It decomposes the marginal gain into conditional expected improvement and improvement probability, providing bounds and under various assumptions, and develops both full-information and data-driven, anytime-valid guarantees for stopping, even with finite data. Empirical validation on credit, employment, and housing datasets shows substantial model multiplicity in disparate impact and modest accuracy loss when minimizing disparity, with the method providing conservative stopping certificates in practice. The framework generalizes beyond LDAs to certify searches over hyperparameters or other model classes, offering a principled, regulator-friendly tool for documenting search adequacy and guiding resource allocation in high-stakes ML pipelines.

Abstract

Recent scholarship has argued that firms building data-driven decision systems in high-stakes domains like employment, credit, and housing should search for "less discriminatory algorithms" (LDAs) (Black et al., 2024). That is, for a given decision problem, firms considering deploying a model should make a good-faith effort to find equally performant models with lower disparate impact across social groups. Evidence from the literature on model multiplicity shows that randomness in training pipelines can lead to multiple models with the same performance, but meaningful variations in disparate impact. This suggests that developers can find LDAs simply by randomly retraining models. Firms cannot continue retraining forever, though, which raises the question: What constitutes a good-faith effort? In this paper, we formalize LDA search via model multiplicity as an optimal stopping problem, where a model developer with limited information wants to produce strong evidence that they have sufficiently explored the space of models. Our primary contribution is an adaptive stopping algorithm that yields a high-probability upper bound on the gains achievable from a continued search, allowing the developer to certify (e.g., to a court) that their search was sufficient. We provide a framework under which developers can impose stronger assumptions about the distribution of models, yielding correspondingly stronger bounds. We validate the method on real-world credit, employment and housing datasets.
Paper Structure (28 sections, 17 theorems, 99 equations, 10 figures, 1 table, 2 algorithms)

This paper contains 28 sections, 17 theorems, 99 equations, 10 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Applications beyond the search for LDAs.
  3. Related work.
  4. Organization of the paper.
  5. Setting and model
  6. Data and utility.
  7. Model distribution.
  8. Adaptive Stopping for Repeated Model Retraining
  9. The full-information regime
  10. The infinite-data regime
  11. The finite-data regime
  12. Data-driven anytime-valid upper bounds on the conditional expected improvement.
  13. Empirical Analysis
  14. Evaluation procedure.
  15. Analysis of model performance.
  16. ...and 13 more sections

Key Result

Lemma 3.1

Let $\{X_t\}_{t=1}^\infty$ be a sequence of iid random variables distributed according to a law $\mathcal{P}_0$. Let $\mathcal{P} \triangleq \mathcal{P}_0^\infty$ be their joint distribution. Let $Y_t$ be the minimum of these variables up to time $t$, i.e., $Y_t \triangleq \min_{s \le t} X_s$. For a Then, the probability over $\mathcal{P}_0$ that $X_{t+1}$ is less than $Y_t$ at any time $t$ is bou

Figures (10)

  • Figure 1: Heatmap of accuracy versus selection rate gap for each dataset and model class. Colors on the plot represent densities of trained models. Each model is trained on a small subset of the data and evaluated on the full population, so accuracy and disparate impact values are population quantities. For many datasets and methods, there is significantly more variation in disparate impact than there is in accuracy: the clusters of models spread horizontally more than they spread vertically. The fact that there are models with significant variation in disparate impact but similar accuracy supports the idea that simply retraining models can yield non-trivially less discriminatory algorithms.
  • Figure 2: \ref{['alg:infinite']} run on several datasets and models. Panel rows are ML methods and panel columns are datasets. In each panel, the horizontal axis is the iteration of the algorithm and the vertical axis is marginal gain. The pink line is our estimated upper bound $\bar{\mu}(\hat{U}_t) \bar{p}_t(0.05)$ and the brown line is the full-information marginal gain. For any $\gamma$, \ref{['alg:infinite']} would stop when the pink line crosses the horizontal line at $\gamma$. Note that the vertical axis is on a log scale.
  • Figure 3: Version of \ref{['fig:acc_vs_srg']} using the fairlearn methods. The disparate impact distribution is shifted to the left as a result of the fairness regularization. Like in \ref{['fig:acc_vs_srg']}, there is substantial variation in disparate impact over the model re-training distribution.
  • Figure 4: \ref{['alg:av3']} run using the same setup as for \ref{['fig:cei']}. For comparison, the upper bound from \ref{['fig:cei']} is shown as a dashed line.
  • Figure 5: \ref{['alg:infinite']} run on fairness-aware methods with the same datasets and base methods as in \ref{['fig:cei']}. For comparison, the marginal gain trends from \ref{['fig:cei']} are shown as dashed lines.
  • ...and 5 more figures

Theorems & Definitions (29)

  • Definition 3.1: $\bar{\mu}$-Bounded CEI for $\mathcal{P}$
  • Lemma 3.1
  • Proposition 3.2
  • Theorem 3.4
  • Theorem 3.6
  • Lemma B.0
  • proof : Proof of \ref{['thm:always-valid-min']}
  • Theorem B.1
  • proof : Proof of \ref{['thm:generic']}
  • Theorem B.2: Ville's inequality
  • ...and 19 more