Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Are There Exceptions to Goodhart's Law? On the Moral Justification of Fairness-Aware Machine Learning

Hilde Weerts, Lambèr Royakkers, Mykola Pechenizkiy

TL;DR

The paper probes the moral foundations of fairness metrics in fair-ml and questions whether treating these metrics as optimization constraints reliably yields fair outcomes. It introduces a moral framework that adds a Utility Space to the existing space-based analysis, distinguishing technical, measurement, and life biases and showing how unjust conditions shape the justified use of fairness metrics like demographic parity and equalized odds. By analyzing the Hardt et al. post-processing approach, the authors demonstrate that constraint-based fairness can misalign with the intended distribution of benefits and harms, potentially harming those it aims to protect. The work argues for a deeper integration of moral reasoning, empirical assumptions, and procedural justice beyond purely technical fixes, and it highlights the need for broader socio-technical interventions to realize fairer AI systems.

Abstract

Fairness-aware machine learning (fair-ml) techniques are algorithmic interventions designed to ensure that individuals who are affected by the predictions of a machine learning model are treated fairly. The problem is often posed as an optimization problem, where the objective is to achieve high predictive performance under a quantitative fairness constraint. However, any attempt to design a fair-ml algorithm must assume a world where Goodhart's law has an exception: when a fairness measure becomes an optimization constraint, it does not cease to be a good measure. In this paper, we argue that fairness measures are particularly sensitive to Goodhart's law. Our main contributions are as follows. First, we present a framework for moral reasoning about the justification of fairness metrics. In contrast to existing work, our framework incorporates the belief that whether a distribution of outcomes is fair, depends not only on the cause of inequalities but also on what moral claims decision subjects have to receive a particular benefit or avoid a burden. We use the framework to distil moral and empirical assumptions under which particular fairness metrics correspond to a fair distribution of outcomes. Second, we explore the extent to which employing fairness metrics as a constraint in a fair-ml algorithm is morally justifiable, exemplified by the fair-ml algorithm introduced by Hardt et al. (2016). We illustrate that enforcing a fairness metric through a fair-ml algorithm often does not result in the fair distribution of outcomes that motivated its use and can even harm the individuals the intervention was intended to protect.

Are There Exceptions to Goodhart's Law? On the Moral Justification of Fairness-Aware Machine Learning

TL;DR

The paper probes the moral foundations of fairness metrics in fair-ml and questions whether treating these metrics as optimization constraints reliably yields fair outcomes. It introduces a moral framework that adds a Utility Space to the existing space-based analysis, distinguishing technical, measurement, and life biases and showing how unjust conditions shape the justified use of fairness metrics like demographic parity and equalized odds. By analyzing the Hardt et al. post-processing approach, the authors demonstrate that constraint-based fairness can misalign with the intended distribution of benefits and harms, potentially harming those it aims to protect. The work argues for a deeper integration of moral reasoning, empirical assumptions, and procedural justice beyond purely technical fixes, and it highlights the need for broader socio-technical interventions to realize fairer AI systems.

Abstract

Fairness-aware machine learning (fair-ml) techniques are algorithmic interventions designed to ensure that individuals who are affected by the predictions of a machine learning model are treated fairly. The problem is often posed as an optimization problem, where the objective is to achieve high predictive performance under a quantitative fairness constraint. However, any attempt to design a fair-ml algorithm must assume a world where Goodhart's law has an exception: when a fairness measure becomes an optimization constraint, it does not cease to be a good measure. In this paper, we argue that fairness measures are particularly sensitive to Goodhart's law. Our main contributions are as follows. First, we present a framework for moral reasoning about the justification of fairness metrics. In contrast to existing work, our framework incorporates the belief that whether a distribution of outcomes is fair, depends not only on the cause of inequalities but also on what moral claims decision subjects have to receive a particular benefit or avoid a burden. We use the framework to distil moral and empirical assumptions under which particular fairness metrics correspond to a fair distribution of outcomes. Second, we explore the extent to which employing fairness metrics as a constraint in a fair-ml algorithm is morally justifiable, exemplified by the fair-ml algorithm introduced by Hardt et al. (2016). We illustrate that enforcing a fairness metric through a fair-ml algorithm often does not result in the fair distribution of outcomes that motivated its use and can even harm the individuals the intervention was intended to protect.
Paper Structure (31 sections, 3 equations, 3 figures)

This paper contains 31 sections, 3 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Running Examples
  4. Two Group Fairness Constraints
  5. Demographic parity
  6. Equalized odds
  7. Egalitarianism
  8. A Framework for the Justification of Fairness Metrics
  9. Framework by Hertweck2021
  10. Technical Bias
  11. Measurement Bias
  12. Life's Bias
  13. The Extension: The Utility Space
  14. Why Causes are Insufficient
  15. The Utility Space
  16. ...and 16 more sections

Figures (3)

  • Figure 1: The relationship between the different spaces and distortion mechanisms. Each mechanism can cause inequality between groups going from one space to the next space.
  • Figure 2: The randomized decision threshold for a probabilistic classifier. The randomized threshold $T_{a}$ is equal to $\underbar{t}_{a}$ with probability $p_{a}$ and equal to $\overline{t}_{a}$ with probability $1 - p_{a}$.
  • Figure 3: ROC curves of randomized decision thresholds.