Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis

Enze Shi, Pankaj Bhagwat, Zhixian Yang, Linglong Kong, Bei Jiang

TL;DR

This work presents a representation-centered fairness framework built on sufficient dimension reduction to balance predictive utility and bias mitigation. By decomposing the feature space into Y-relevant, Z-sensitive, and shared subspaces and progressively removing shared information, the method achieves controlled fairness without sacrificing accuracy. The authors provide a utility–fairness decomposition, an influence-function-based analysis of estimator behavior, and an algorithmic pipeline for sequential post-SDR training, supported by simulations and real-data experiments. The results show robust improvements in fairness metrics with competitive predictive performance, highlighting practical applicability in high-stakes domains. A key contribution is the explicit linking of SDR-based subspace manipulation with asymptotic guarantees and measurable fairness through distance covariance.

Abstract

Machine learning models have achieved widespread success but often inherit and amplify historical biases, resulting in unfair outcomes. Traditional fairness methods typically impose constraints at the prediction level, without addressing underlying biases in data representations. In this work, we propose a principled framework that adjusts data representations to balance predictive utility and fairness. Using sufficient dimension reduction, we decompose the feature space into target-relevant, sensitive, and shared components, and control the fairness-utility trade-off by selectively removing sensitive information. We provide a theoretical analysis of how prediction error and fairness gaps evolve as shared subspaces are added, and employ influence functions to quantify their effects on the asymptotic behavior of parameter estimates. Experiments on both synthetic and real-world datasets validate our theoretical insights and show that the proposed method effectively improves fairness while preserving predictive performance.

Understanding Fairness and Prediction Error through Subspace Decomposition and Influence Analysis

TL;DR

This work presents a representation-centered fairness framework built on sufficient dimension reduction to balance predictive utility and bias mitigation. By decomposing the feature space into Y-relevant, Z-sensitive, and shared subspaces and progressively removing shared information, the method achieves controlled fairness without sacrificing accuracy. The authors provide a utility–fairness decomposition, an influence-function-based analysis of estimator behavior, and an algorithmic pipeline for sequential post-SDR training, supported by simulations and real-data experiments. The results show robust improvements in fairness metrics with competitive predictive performance, highlighting practical applicability in high-stakes domains. A key contribution is the explicit linking of SDR-based subspace manipulation with asymptotic guarantees and measurable fairness through distance covariance.

Abstract

Machine learning models have achieved widespread success but often inherit and amplify historical biases, resulting in unfair outcomes. Traditional fairness methods typically impose constraints at the prediction level, without addressing underlying biases in data representations. In this work, we propose a principled framework that adjusts data representations to balance predictive utility and fairness. Using sufficient dimension reduction, we decompose the feature space into target-relevant, sensitive, and shared components, and control the fairness-utility trade-off by selectively removing sensitive information. We provide a theoretical analysis of how prediction error and fairness gaps evolve as shared subspaces are added, and employ influence functions to quantify their effects on the asymptotic behavior of parameter estimates. Experiments on both synthetic and real-world datasets validate our theoretical insights and show that the proposed method effectively improves fairness while preserving predictive performance.

Paper Structure

This paper contains 28 sections, 8 theorems, 64 equations, 2 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Motivation
  5. Problem Setup
  6. Utility and Fairness Trade-off
  7. Subspace Estimation and Algorithm Implementation
  8. Estimation of Projections
  9. Algorithm Implementation
  10. Algorithm Implementation
  11. Theoretical Analysis
  12. Influence Functions
  13. Asymptotic Normality of Estimators
  14. Experiments
  15. Simulation Studies
  16. ...and 13 more sections

Key Result

Theorem 3.1

Let $\widetilde{B}$ be an orthonormal matrix satisfying condition sdr-B, then $\operatorname{Span}(\widetilde{B}) \subseteq \operatorname{Span}(Q_z B)= \mathcal{S}_{\boldsymbol{Y} \mid \boldsymbol{X}} \cap \mathcal{S}_{\boldsymbol{Z} \mid \boldsymbol{X}}^{\perp}$.

Figures (2)

  • Figure 1: Left panel: Trends of RMSE and parameter distance as the number of shared dimensions increases, averaged over 30 replications. Right panel: Distributional discrepancy between sensitive groups visualized via LDA, along with the average Wasserstein distance (WD) across all $p$ dimensions between the original and projected data as the shared dimension increases in one replication.
  • Figure 2: Trends of RMSE, parameter distance and distributional discrepancy as the number of shared dimensions increases when the linear SDR assumption is violated.

Theorems & Definitions (17)

  • Theorem 3.1
  • Theorem 3.2
  • Theorem 3.3
  • Theorem 4.1
  • Remark 4.2
  • Remark 4.3
  • Lemma 5.1
  • Theorem 5.2
  • Corollary 5.3
  • Theorem 5.4
  • ...and 7 more