Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Fairness-Performance Pareto Front Computation

Mark Kozdoba, Binyamin Perets, Shie Mannor

TL;DR

This paper addresses computing the optimal fairness-performance Pareto front for representations without training complex fair models. It develops a theory (Factorization Lemma and Invertibility Theorem) that reduces the problem to a small discrete space and introduces MIFPO, a model-independent concave minimization problem with linear fairness constraints solved via DC programming to obtain the Pareto front. The approach provides a principled, scalable benchmark for evaluating fair representation methods and yields an experimental Pareto frontier on real datasets that often dominates existing fair classifiers. Overall, the work offers a practical, theoretically grounded framework to quantify and compare fairness-performance trade-offs in representation learning.

Abstract

There is a well known intrinsic trade-off between the fairness of a representation and the performance of classifiers derived from the representation. Due to the complexity of optimisation algorithms in most modern representation learning approaches, for a given method it may be non-trivial to decide whether the obtained fairness-performance curve of the method is optimal, i.e., whether it is close to the true Pareto front for these quantities for the underlying data distribution. In this paper we propose a new method to compute the optimal Pareto front, which does not require the training of complex representation models. We show that optimal fair representations possess several useful structural properties, and that these properties enable a reduction of the computation of the Pareto Front to a compact discrete problem. We then also show that these compact approximating problems can be efficiently solved via off-the shelf concave-convex programming methods. Since our approach is independent of the specific model of representations, it may be used as the benchmark to which representation learning algorithms may be compared. We experimentally evaluate the approach on a number of real world benchmark datasets.

Efficient Fairness-Performance Pareto Front Computation

TL;DR

This paper addresses computing the optimal fairness-performance Pareto front for representations without training complex fair models. It develops a theory (Factorization Lemma and Invertibility Theorem) that reduces the problem to a small discrete space and introduces MIFPO, a model-independent concave minimization problem with linear fairness constraints solved via DC programming to obtain the Pareto front. The approach provides a principled, scalable benchmark for evaluating fair representation methods and yields an experimental Pareto frontier on real datasets that often dominates existing fair classifiers. Overall, the work offers a practical, theoretically grounded framework to quantify and compare fairness-performance trade-offs in representation learning.

Abstract

There is a well known intrinsic trade-off between the fairness of a representation and the performance of classifiers derived from the representation. Due to the complexity of optimisation algorithms in most modern representation learning approaches, for a given method it may be non-trivial to decide whether the obtained fairness-performance curve of the method is optimal, i.e., whether it is close to the true Pareto front for these quantities for the underlying data distribution. In this paper we propose a new method to compute the optimal Pareto front, which does not require the training of complex representation models. We show that optimal fair representations possess several useful structural properties, and that these properties enable a reduction of the computation of the Pareto Front to a compact discrete problem. We then also show that these compact approximating problems can be efficiently solved via off-the shelf concave-convex programming methods. Since our approach is independent of the specific model of representations, it may be used as the benchmark to which representation learning algorithms may be compared. We experimentally evaluate the approach on a number of real world benchmark datasets.
Paper Structure (26 sections, 10 theorems, 92 equations, 5 figures, 1 algorithm)

This paper contains 26 sections, 10 theorems, 92 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature and Prior Work
  3. Structure of Fair Representations
  4. Problem Setting
  5. Classification and Factorization
  6. The Invertibility Theorem
  7. The Model Independent Optimization Problem
  8. MIFPO Definition
  9. The Full Algorithm
  10. Experiments
  11. Implementation
  12. Conclusions, Limitations, And Future Work
  13. Monotonicity of Loss Under Representations
  14. MIFPO and Optimal Transport
  15. Proof Of Theorem \ref{['lem:invertibility_lemma']}
  16. ...and 11 more sections

Key Result

Lemma 3.1

Let $\hat{Y}$ be a classifier of $Y$, let the representation uncertainty measure be given by eq:accuraccy_h_definition. Then there is a representation given by a random variable $Z$ on a set $\mathcal{Z}$ with $\left| \mathcal{Z} \right| = 2$, such that Conversely, for any given representation $Z$, there is a classifier $\hat{Y}$ of $Y$ as a function of $Z$ (and thus of $(X,A)$), such that

Figures (5)

  • Figure 1: Fair Representation Problem Setting.
  • Figure 2: (a) The MIFPO Setting (b) Distribution of $P(Y=1|X,A)$ for each group across datasets.
  • Figure 3: Comparison of fairness-accuracy trade-offs across three benchmark datasets: Health (left), ACSIncome-CA (middle), and ACSIncome-US (right). MIFPO's Pareto front is represented as a solid line with markers. The horizontal axis represents the fairness constraint (statistical parity distance), while the vertical axis shows prediction accuracy.
  • Figure 4: Comparing common fair classification pipelines to the MIFPO Pareto front, for LSAC, COMPAS, and ADULT datasets. For FGBM and LGBM+Post Process methods, each point represents a trade-off obtained at a single hyper-parameter configuration.
  • Figure 5: Comparing FairFront to MIFPO accuracy-fairness tradeoff on two curated datasets.

Theorems & Definitions (19)

  • Lemma 3.1
  • Lemma 3.2: Factorization
  • Theorem 3.1
  • Definition 4.1: MIFPO
  • Lemma A.1
  • proof
  • proof : Proof Of Theorem \ref{['lem:invertibility_lemma']}
  • Lemma D.1
  • proof
  • Lemma E.1
  • ...and 9 more