Bayes-Optimal Classifiers under Group Fairness

TL;DR

The paper introduces a unified Neyman-Pearson-based framework to derive Bayes-optimal classifiers under group fairness constraints and proposes FairBayes, a fast post-processing method that achieves direct control of disparity via group-wise thresholds. It proves that, under demographic parity with tolerance $\delta$, the Bayes-optimal solution is a group-wise threshold rule, and provides a practical algorithm to estimate thresholds from data. The approach yields favorable fairness-accuracy tradeoffs in synthetic and real-data experiments while maintaining computational efficiency. This work offers a principled path to explicitly constrain unfairness levels while preserving near-optimal predictive performance, with potential extensions to multi-class protected attributes and other fairness criteria.

Abstract

Machine learning algorithms are becoming integrated into more and more high-stakes decision-making processes, such as in social welfare issues. Due to the need of mitigating the potentially disparate impacts from algorithmic predictions, many approaches have been proposed in the emerging area of fair machine learning. However, the fundamental problem of characterizing Bayes-optimal classifiers under various group fairness constraints has only been investigated in some special cases. Based on the classical Neyman-Pearson argument (Neyman and Pearson, 1933; Shao, 2003) for optimal hypothesis testing, this paper provides a unified framework for deriving Bayes-optimal classifiers under group fairness. This enables us to propose a group-based thresholding method we call FairBayes, that can directly control disparity, and achieve an essentially optimal fairness-accuracy tradeoff. These advantages are supported by thorough experiments.

Figures (5)

• Figure 1: Difference in demographic parity (DDP) and misclassification rate (MCR) for four classifiers. Panel (a): the unconstrained Bayes-optimal classifier. Panel (b): a $\delta$-fair Bayes-optimal classifier. Panel (c): fair Bayes-optimal classifier. Panel (d): non-optimal fair classifier. In the plots, the red and blue lines are the conditional probabilities of $Y=1$ in the two groups; the horizontal red and blue dashed lines are the thresholds for the groups (classifying as $1$ for $x$ less than a vertical dashed line); the light green curve is $x\mapsto p(x)$, the density function of $X$; the dark green curve is $x\mapsto p(x)P(\widehat{Y}\neq Y|X=x)$, whose integral is the misclassification rate. The DDP and misclassification rate are given by the shaded area in orange and yellow, respectively.
• Figure 2: Fairness-Accuracy tradeoff of our classifier and the fair Bayes-optimal classifier with $p=10$, $\sigma=1$. Panel (a): Tradeoff between accuracy and demographic parity. Panel (b): Tradeoff between accuracy and equality of opportunity parity.
• Figure 3: Fairness-Accuracy tradeoff on "Adult" dataset. Left panel: Tradeoff with respect to demographic parity. Right panel: Tradeoff with respect to Equality of opportunity.
• Figure 4: Fairness-Accuracy tradeoff on the "COMPAS" dataset. Left panel: Tradeoff with respect to demographic parity (DDP). Right panel: Tradeoff with respect to Equality of opportunity (DEO).
• Figure 5: Fairness-Accuracy tradeoff on the "Law school" dataset. Left panel: Tradeoff with respect to demographic parity (DDP). Right panel: Tradeoff with respect to Equality of opportunity (DEO).

Theorems & Definitions (21)

• Definition 3.1: Randomized classifier
• Definition 3.2: Demographic Parity
• Proposition 4.1
• Theorem 4.2
• Remark 4.3
• Lemma A.1
• proof
• Theorem C.1: Fair Bayes-optimal Classifiers with Cost-sensitive Risk
• proof
• Theorem D.1