Deep Fair Learning: A Unified Framework for Fine-tuning Representations with Sufficient Networks

TL;DR

Deep Fair Learning is proposed, a framework that integrates nonlinear sufficient dimension reduction with deep learning to construct fair and informative representations that achieves a superior balance between fairness and utility, significantly outperforming state-of-the-art baselines.

Abstract

Ensuring fairness in machine learning is a critical and challenging task, as biased data representations often lead to unfair predictions. To address this, we propose Deep Fair Learning, a framework that integrates nonlinear sufficient dimension reduction with deep learning to construct fair and informative representations. By introducing a novel penalty term during fine-tuning, our method enforces conditional independence between sensitive attributes and learned representations, addressing bias at its source while preserving predictive performance. Unlike prior methods, it supports diverse sensitive attributes, including continuous, discrete, binary, or multi-group types. Experiments on various types of data structure show that our approach achieves a superior balance between fairness and utility, significantly outperforming state-of-the-art baselines.

Figures (4)

• Figure 1: Fair and sufficient subspace. The surface represents the dependence of $\boldsymbol{Z}$ on $\boldsymbol{X}$, where changes in $\mathcal{S}_1$ do not affect $\boldsymbol{Z}$.
• Figure 2: The flowchart of Deep Fair Learning framework.
• Figure 3: DFL fine-tuning performance on Adult and Bank
• Figure 4: DFL fine-tuning performance on BIOS and MOJI.

Theorems & Definitions (3)

• Definition 3.1: Independence
• Definition 3.2: Separation
• Remark 4.1