Simple and Effective Specialized Representations for Fair Classifiers

TL;DR

Fair classification is challenged by unstable adversarial training and costly distribution matching. The authors introduce a CFD-based framework that learns fair, specialized representations by aligning conditional distributions to a Gaussian target, avoiding adversarial training. They present two variants: FmCF for general fair representation learning and FmSS for simplified, moment-based guarantees in classification. Empirical results on standard benchmarks show competitive accuracy with improved fairness, and the methods do not require sensitive attributes at inference, highlighting practicality and privacy advantages.

Abstract

Fair classification is a critical challenge that has gained increasing importance due to international regulations and its growing use in high-stakes decision-making settings. Existing methods often rely on adversarial learning or distribution matching across sensitive groups; however, adversarial learning can be unstable, and distribution matching can be computationally intensive. To address these limitations, we propose a novel approach based on the characteristic function distance. Our method ensures that the learned representation contains minimal sensitive information while maintaining high effectiveness for downstream tasks. By utilizing characteristic functions, we achieve a more stable and efficient solution compared to traditional methods. Additionally, we introduce a simple relaxation of the objective function that guarantees fairness in common classification models with no performance degradation. Experimental results on benchmark datasets demonstrate that our approach consistently matches or achieves better fairness and predictive accuracy than existing methods. Moreover, our method maintains robustness and computational efficiency, making it a practical solution for real-world applications.

Figures (12)

• Figure 1: Overview of the proposed approach. Each sample $X$ is associated with a sensitive attribute $S$ and a target label $Y$. The conditional distribution $\mathbb{P}_{X|S}$ is mapped to a new distribution $\mathbb{P}_{Z \mid S}$, which is encouraged to resemble a Gaussian Distribution via the CFD. The encoded representation $Z$ minimizes $\Delta(\mathbb{P}_{Z \mid S_0}, \mathbb{P}_{Z \mid S_1})$ while retaining task-relevant information.
• Figure 2: The closer $\mathbb{E}[X | y = 0]$ is to $\mathbb{E}[X | y = 1]$, the less predictive power the feature has, causing the LR coefficient $\beta$ to approach 0.
• Figure 3: Latent distributions of $\mathbb{P}_{Z \mid S}$ and $\mathbb{P}_{Z|Y}$ obtained using FmCF.
• Figure 4: Pareto front between FmCF and FNF comparing task accuracy and fairness (95% confidence intervals from 10 seeds).
• Figure 5: Pareto varying latent dimension on Health dataset.

Theorems & Definitions (6)

• Theorem 5.1
• Theorem 5.2
• Theorem 5.3
• proof
• proof
• proof