Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Unifying Group Fairness Evaluation from a Sparsity Perspective

Zhecheng Sheng, Jiawei Zhang, Enmao Diao

TL;DR

Addressing the challenge of generalizable fairness evaluation across diverse ML problems, the paper reframes algorithmic fairness through distributional sparsity. It builds a unified framework around the PQ Index $\mathbf{I}_{p,q}(\bm{w})$ and connects it to the Maximum Pairwise Difference $MPD$ and the Gini Index, enabling compatibility with Statistical Parity and Equalized Odds across classification and regression. The authors prove theoretical properties of the PQ Index, show bounds relating sparsity measures, and demonstrate alignment with existing fairness criteria in extensive experiments on multiple datasets and bias-mitigation methods, including intersectional settings. They also discuss dataset limitations, broader societal impact, and ethical considerations, highlighting the practical value of sparsity-based fairness for broad AI applications.

Abstract

Ensuring algorithmic fairness remains a significant challenge in machine learning, particularly as models are increasingly applied across diverse domains. While numerous fairness criteria exist, they often lack generalizability across different machine learning problems. This paper examines the connections and differences among various sparsity measures in promoting fairness and proposes a unified sparsity-based framework for evaluating algorithmic fairness. The framework aligns with existing fairness criteria and demonstrates broad applicability to a wide range of machine learning tasks. We demonstrate the effectiveness of the proposed framework as an evaluation metric through extensive experiments on a variety of datasets and bias mitigation methods. This work provides a novel perspective to algorithmic fairness by framing it through the lens of sparsity and social equity, offering potential for broader impact on fairness research and applications.

Toward Unifying Group Fairness Evaluation from a Sparsity Perspective

TL;DR

Addressing the challenge of generalizable fairness evaluation across diverse ML problems, the paper reframes algorithmic fairness through distributional sparsity. It builds a unified framework around the PQ Index and connects it to the Maximum Pairwise Difference and the Gini Index, enabling compatibility with Statistical Parity and Equalized Odds across classification and regression. The authors prove theoretical properties of the PQ Index, show bounds relating sparsity measures, and demonstrate alignment with existing fairness criteria in extensive experiments on multiple datasets and bias-mitigation methods, including intersectional settings. They also discuss dataset limitations, broader societal impact, and ethical considerations, highlighting the practical value of sparsity-based fairness for broad AI applications.

Abstract

Ensuring algorithmic fairness remains a significant challenge in machine learning, particularly as models are increasingly applied across diverse domains. While numerous fairness criteria exist, they often lack generalizability across different machine learning problems. This paper examines the connections and differences among various sparsity measures in promoting fairness and proposes a unified sparsity-based framework for evaluating algorithmic fairness. The framework aligns with existing fairness criteria and demonstrates broad applicability to a wide range of machine learning tasks. We demonstrate the effectiveness of the proposed framework as an evaluation metric through extensive experiments on a variety of datasets and bias mitigation methods. This work provides a novel perspective to algorithmic fairness by framing it through the lens of sparsity and social equity, offering potential for broader impact on fairness research and applications.

Paper Structure

This paper contains 47 sections, 7 theorems, 59 equations, 14 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sparsity Measures.
  4. Algorithmic Fairness.
  5. Bias Mitigation.
  6. Sparsity
  7. Relation to Fairness.
  8. Properties of PQ Index
  9. A Comparison among the Sparsity Measures
  10. Unifying Group Fairness with Sparsity
  11. Statistical Parity (SP)
  12. Classification.
  13. Regression.
  14. Equalized Odds (EO)
  15. Classification.
  16. ...and 32 more sections

Key Result

Theorem 3.1

For $\bm{w}$, if there exists $k\in\{1,\dots,d\}$ such that $w_k\neq 0$ and $w_j =0$ ($j\neq k$), then:

Figures (14)

  • Figure 1: The plots of (a) $Gini(\bm{w})$ and (b) $\mathbf{I}_{1, 2}(\bm{w})$ for $d=3$ and $\|\bm{w}\|_1=1$. In each plot, the horizontal axes correspond to $w_1$ and $w_2$ (where $w_3=1-w_1-w_2$). The vertical axis shows the value of Gini Index or PQ Index. Since there are 6 possible permutations of $[w_1, w_2, w_3]$, $Gini(\bm{w})$ is composed of subsets of 6 distinct planes. In contrast, $\mathbf{I}_{1, 2}(\bm{w})$ has a smooth surface. Both Gini Index and PQ Index attain their minimum at $\bm{w}=[3^{-1},3^{-1},3^{-1}]^{ \mathrm{ T} }$.
  • Figure 2: Comparison of sparsity criteria with baseline criteria in two classification dataset. The top row shows results from baseline criteria; the bottom row shows results from the proposed sparsity criteria. The x-axis of each plot represents the value of various criteria.
  • Figure 3: Comparison in Community & Crimes.
  • Figure 4: Top: Simulated binary classfication dataset. Bottom: Adult dataset. Legend indicates bias mitigation methods, with subscripts denoting hyperparameter.
  • Figure 5: Results of applying $\bm{w} = \exp({\bm{w}})$ to ensure positivity in different dataset and metrics.
  • ...and 9 more figures

Theorems & Definitions (33)

  • Definition 3.1: Maximum Pairwise Difference
  • Definition 3.2: Gini Index
  • Definition 3.3: PQ Index
  • Theorem 3.1
  • Remark 3.1
  • Theorem 3.2
  • Remark 3.2
  • Theorem 3.3
  • Remark 3.3
  • Theorem 3.4
  • ...and 23 more