FAIRM: Learning invariant representations for algorithmic fairness and domain generalization with minimax optimality
Sai Li, Linjun Zhang
TL;DR
FAIRM tackles distribution shifts by enforcing invariance and fairness across training environments to achieve robust OOD performance. It introduces a full-information invariant oracle and a training-environment counterpart, with a special focus on a diversity-type condition that enables recovery of the full-information benchmark from training data. The paper provides finite-sample guarantees for empirical FAIRM, develops a computationally efficient linear-model algorithm with minimax-optimal domain generalization, and demonstrates superior performance on synthetic data and Color MNIST. Its theoretical guarantees encompass both domain generalization and multi-calibration, offering a practical, distribution-free approach to fair and generalizable learning. The work highlights FAIRM’s potential extensions to nonlinear representations and broader invariant-learning paradigms.
Abstract
Machine learning methods often assume that the test data have the same distribution as the training data. However, this assumption may not hold due to multiple levels of heterogeneity in applications, raising issues in algorithmic fairness and domain generalization. In this work, we address the problem of fair and generalizable machine learning by invariant principles. We propose a training environment-based oracle, FAIRM, which has desirable fairness and domain generalization properties under a diversity-type condition. We then provide an empirical FAIRM with finite-sample theoretical guarantees under weak distributional assumptions. We then develop efficient algorithms to realize FAIRM in linear models and demonstrate the nonasymptotic performance with minimax optimality. We evaluate our method in numerical experiments with synthetic data and MNIST data and show that it outperforms its counterparts.