Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule

Xuan Zhao, Klaus Broelemann, Salvatore Ruggieri, Gjergji Kasneci

TL;DR

An innovative approach to enhancing the empirical risk minimization (ERM) process in model training through a refined reweighting scheme of the training data to enhance fairness is introduced, revealing a consistent improvement in the balance between prediction performance and fairness metrics across various experiments.

Abstract

We introduce an innovative approach to enhancing the empirical risk minimization (ERM) process in model training through a refined reweighting scheme of the training data to enhance fairness. This scheme aims to uphold the sufficiency rule in fairness by ensuring that optimal predictors maintain consistency across diverse sub-groups. We employ a bilevel formulation to address this challenge, wherein we explore sample reweighting strategies. Unlike conventional methods that hinge on model size, our formulation bases generalization complexity on the space of sample weights. We discretize the weights to improve training speed. Empirical validation of our method showcases its effectiveness and robustness, revealing a consistent improvement in the balance between prediction performance and fairness metrics across various experiments.

Enhancing Fairness through Reweighting: A Path to Attain the Sufficiency Rule

TL;DR

An innovative approach to enhancing the empirical risk minimization (ERM) process in model training through a refined reweighting scheme of the training data to enhance fairness is introduced, revealing a consistent improvement in the balance between prediction performance and fairness metrics across various experiments.

Abstract

We introduce an innovative approach to enhancing the empirical risk minimization (ERM) process in model training through a refined reweighting scheme of the training data to enhance fairness. This scheme aims to uphold the sufficiency rule in fairness by ensuring that optimal predictors maintain consistency across diverse sub-groups. We employ a bilevel formulation to address this challenge, wherein we explore sample reweighting strategies. Unlike conventional methods that hinge on model size, our formulation bases generalization complexity on the space of sample weights. We discretize the weights to improve training speed. Empirical validation of our method showcases its effectiveness and robustness, revealing a consistent improvement in the balance between prediction performance and fairness metrics across various experiments.
Paper Structure (34 sections, 1 theorem, 14 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 34 sections, 1 theorem, 14 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Preliminaries and Related Work
  2. Sufficiency Rule in Fairness
  3. Sufficiency Gap
  4. Relation to Other Fairness Notions
  5. Invariant Risk Minimization
  6. Reweighting
  7. Reweighting to Achieve Sufficiency Rule
  8. Bilevel Formulation of Reweighting
  9. Connection to the Sufficiency Rule:
  10. Enhance Reweighting by Sparsity and Continuation
  11. Optimization Method
  12. Experiments
  13. Baselines
  14. Datasets and Experiment Setups
  15. Analysis
  16. ...and 19 more sections

Key Result

Proposition 1

In a classification task, minimizing the loss in the outer loop as illustrated in details in Section sec:IRM is tantamount to: where $h^*_0$, $h^*_1$ are the optimal predictor for each sub-group, $h^*=h^*_1=h^*_0$ and $z=\phi(x)$.

Figures (4)

  • Figure 1: Change of accuracy and sufficiency gap under different noise ratios on Toxic comments and CelebA datasets, which shows that our method is robust when the data label is noisy.
  • Figure 2: The evolution of probability score distribution during the search process reveals a trend where most probabilities tend to converge towards either 0 or 1. This convergence ultimately leads to deterministic weights and convergence of the algorithm.
  • Figure 3: Choices of $K$ (selected set size). The size is set to 5000 for Toxic comments and 10000 for CelebA.
  • Figure 4: The fluctuation in the distribution of group weight fractions for ResNet-18 on the CelebA dataset is notable. Specifically, there's a shift to 20% for both the (Male, Blond Hair) and (Male, Dark Hair) gourps. Similarly, the (Female, Dark Hair) and (Female, Blond Hair) groups see their fractions adjusted to 30%. These observations suggest that our methodology is capable of autonomously adapting weight fractions across various (sub)-groups.

Theorems & Definitions (1)

  • Proposition 1