The Statistical Fairness-Accuracy Frontier

TL;DR

This work analyzes the fairness-accuracy frontier in a finite-sample regime for two-group regression, extending the population-frontier perspective to practical data-limited settings. It identifies minimax-optimal estimators under both known and unknown covariances, deriving explicit risk bounds and optimal sampling rules that reflect group heterogeneity. The results show that finite-sample effects shift the anticipated fairness-accuracy trade-offs, inducing asymmetric impacts across groups and guiding allocation of sampling resources. A uniform frontier bound provides high-probability guarantees for the entire FA frontier, enabling policymaking and deployment decisions based on empirical frontiers with quantified uncertainty.

Abstract

Machine learning models must balance accuracy and fairness, but these goals often conflict, particularly when data come from multiple demographic groups. A useful tool for understanding this trade-off is the fairness-accuracy (FA) frontier, which characterizes the set of models that cannot be simultaneously improved in both fairness and accuracy. Prior analyses of the FA frontier provide a full characterization under the assumption of complete knowledge of population distributions -- an unrealistic ideal. We study the FA frontier in the finite-sample regime, showing how it deviates from its population counterpart and quantifying the worst-case gap between them. In particular, we derive minimax-optimal estimators that depend on the designer's knowledge of the covariate distribution. For each estimator, we characterize how finite-sample effects asymmetrically impact each group's risk, and identify optimal sample allocation strategies. Our results transform the FA frontier from a theoretical construct into a practical tool for policymakers and practitioners who must often design algorithms with limited data.

Figures (5)

• Figure 1: (a) Population FA frontier: The blue shaded area represents the set of risk pairs that are achievable by some linear model, and $\beta_r$ and $\beta_b$ are the error-minimizing models for groups $r$ and $b$, respectively. $\beta_\text{fair}$ is the error-equalizing model, which falls between $\beta_r$ and $\beta_b$ in the group-balanced case. The FA frontier corresponds to the purple region along the boundary of the feasible set of error pairs. (b) For any $\lambda \in [0,1]$, $\beta_\lambda$ is the first point of tangency between the line $\lambda x + (1-\lambda) y=c$ and the FA frontier as we increase $c>0$. This point moves along the frontier from $\beta_r$ to $\beta_b$ as $\lambda$ ranges from $1 \to 0$.
• Figure 2: (a) Finite-sample estimation: The error pair corresponding to the empirical estimator $\widehat{\beta}_\lambda$ lies on the line $\lambda x + (1 - \lambda) y = c'$, where $c - c'$ is the excess risk. (b) Asymmetric group-wise impact: The displacement from $\beta_\lambda$ to $\widehat{\beta}_\lambda$ decomposes into a vertical change in the blue-group error and a horizontal change in the red-group error; their unequal magnitudes show that the estimator affects the two groups asymmetrically.
• Figure 3: Group-Balanced vs. Group-Skewed. To avoid cluttering the figure, we label points by their corresponding predictors; for example, $f_r$ denotes the point $(\mathcal{R}_r(f_r), \mathcal{R}_b(f_r))$.
• Figure 4: Illustration of FA frontier, parametrized by $\lambda$, and its empirical version.
• Figure 5: We sweep $\lambda$ on a uniform grid of $50$ points in $[0,1]$. For each $\lambda$, we run 100 Monte Carlo repetitions; in each iteration, we draw data $\mathcal{S}_g = \{(X_g, Y_g)\}_{i = 1}^{n_g}$ from the group-$g$ Gaussian linear model with $\sigma^2 = 1$, build the estimators $\tilde{\beta}_\lambda, \widehat{\beta}_\lambda$, and calculate the corresponding risk pair $(\mathcal{R}_r, \mathcal{R}_b)$ for each estimator. We plot the mean empirical frontiers across repetitions and depict the empirical contraction for the choice of $\lambda = 0.5$ for two regimes: (a) Equal samples:$n_r = n_b = 30, d = 12, \rho_r = 1, \rho_b = 2.5$, (b) Equal covariance:$n_r = 50, n_b = 30, d = 12, \rho_r = \rho_b = 1.75$.

Theorems & Definitions (30)

• Definition 1: Fairness-Accuracy (FA) Dominance
• Definition 2: Fairness-Accuracy (FA) Frontier
• Definition 3: Linear Model Class
• Lemma 1: Group-Balanced Structure
• Remark 1: Role of Concentration and Anticoncentration
• Proposition 1
• Theorem 1
• Remark 2
• Corollary 1
• Corollary 2