Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium

Mehdi Yazdani-Jahromi, Ali Khodabandeh Yalabadi, AmirArsalan Rajabi, Aida Tayebi, Ivan Garibay, Ozlem Ozmen Garibay

TL;DR

A novel methodology grounded in bilevel optimization principles that concurrently optimizes for both accuracy and fairness objectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model is proposed.

Abstract

The persistent challenge of bias in machine learning models necessitates robust solutions to ensure parity and equal treatment across diverse groups, particularly in classification tasks. Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness. To address this, we propose a novel methodology grounded in bilevel optimization principles. Our deep learning-based approach concurrently optimizes for both accuracy and fairness objectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model. Theoretical analysis indicates that the upper bound on the loss incurred by this method is less than or equal to the loss of the Lagrangian approach, which involves adding a regularization term to the loss function. We demonstrate the efficacy of our model primarily on tabular datasets such as UCI Adult and Heritage Health. When benchmarked against state-of-the-art fairness methods, our model exhibits superior performance, advancing fairness-aware machine learning solutions and bridging the accuracy-fairness gap. The implementation of FairBiNN is available on https://github.com/yazdanimehdi/FairBiNN.

Fair Bilevel Neural Network (FairBiNN): On Balancing fairness and accuracy via Stackelberg Equilibrium

TL;DR

A novel methodology grounded in bilevel optimization principles that concurrently optimizes for both accuracy and fairness objectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model is proposed.

Abstract

The persistent challenge of bias in machine learning models necessitates robust solutions to ensure parity and equal treatment across diverse groups, particularly in classification tasks. Current methods for mitigating bias often result in information loss and an inadequate balance between accuracy and fairness. To address this, we propose a novel methodology grounded in bilevel optimization principles. Our deep learning-based approach concurrently optimizes for both accuracy and fairness objectives, and under certain assumptions, achieving proven Pareto optimal solutions while mitigating bias in the trained model. Theoretical analysis indicates that the upper bound on the loss incurred by this method is less than or equal to the loss of the Lagrangian approach, which involves adding a regularization term to the loss function. We demonstrate the efficacy of our model primarily on tabular datasets such as UCI Adult and Heritage Health. When benchmarked against state-of-the-art fairness methods, our model exhibits superior performance, advancing fairness-aware machine learning solutions and bridging the accuracy-fairness gap. The implementation of FairBiNN is available on https://github.com/yazdanimehdi/FairBiNN.

Paper Structure

This paper contains 48 sections, 10 theorems, 50 equations, 10 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related works
  3. Methodology
  4. Theoretical Analysis
  5. Problem Formulation
  6. Theoretical Foundation: Stackelberg Equilibrium and Pareto Optimality
  7. Practical Implementation
  8. Experiments
  9. Datasets:
  10. Baselines:
  11. Bilevel (FairBiNN) vs. Lagrangian Method
  12. Benchmark Comparison
  13. Limitations and Future Work
  14. Discussion and Conclusion
  15. Appendix / supplemental material
  16. ...and 33 more sections

Key Result

Lemma 3.5

Let $f(x;\theta)$ be a neural network with L layers, where each layer is a linear transformation followed by a Lipschitz continuous activation function. Let $\theta$ be the set of all parameters of the neural network, and $\theta_s \subseteq \theta$ be any subset of parameters. Then, $f(x; \theta)$

Figures (10)

  • Figure 1: Accuracy of various benchmark models compared to the FairBiNN model versus statistical demographic parity for the (a) UCI Adult dataset and (b) Heritage Health dataset. The optimal region on this graph is the bottom right, indicating high accuracy and low DP. The results demonstrate that our model (red diamond markers) significantly outperforms other benchmark models on the UCI Adult dataset and closely competes with recent state-of-the-art models on the Heritage Health dataset.
  • Figure 2: BCE loss over epochs for the Lagrangian, Bi-level, and Without Fairness approaches on (a) the Adult dataset and (b) the Health dataset. These results illustrate that the Bi-level optimization framework achieves lower BCE loss compared to the Lagrangian approach in these experiments, highlighting its potential in optimizing both accuracy and fairness objectives in neural networks.
  • Figure 3: Comparison of FairBiNN and Lagrangian methods on UCI Adult and Heritage Health datasets
  • Figure 4: Attractive Attribute of CelebA Dataset as the Target Attribute. (a) reflects the trade-off between Average Precision and Demographic Parity Difference. (b) shows the trade-off between Average Precision and Equalized Odds Difference.
  • Figure 5: Smiling Attribute of CelebA Dataset as the Target Attribute. (a) reflects the trade-off between Average Precision and Demographic Parity Difference. (b) shows the trade-off between Average Precision and Equalized Odds Difference.
  • ...and 5 more figures

Theorems & Definitions (18)

  • Definition 3.1
  • Lemma 3.5
  • Theorem 3.6
  • Theorem 3.7
  • Theorem 3.8
  • proof
  • Theorem 3.9
  • proof
  • Theorem 3.10
  • Lemma A.4
  • ...and 8 more