Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Unified View of Group Fairness Tradeoffs Using Partial Information Decomposition

Faisal Hamman, Sanghamitra Dutta

TL;DR

A novel information-theoretic perspective on the relationship between prominent group fairness notions in machine learning, namely statistical parity, equalized odds, and predictive parity is introduced by leveraging a body of work in information theory called partial information decomposition (PID).

Abstract

This paper introduces a novel information-theoretic perspective on the relationship between prominent group fairness notions in machine learning, namely statistical parity, equalized odds, and predictive parity. It is well known that simultaneous satisfiability of these three fairness notions is usually impossible, motivating practitioners to resort to approximate fairness solutions rather than stringent satisfiability of these definitions. However, a comprehensive analysis of their interrelations, particularly when they are not exactly satisfied, remains largely unexplored. Our main contribution lies in elucidating an exact relationship between these three measures of (un)fairness by leveraging a body of work in information theory called partial information decomposition (PID). In this work, we leverage PID to identify the granular regions where these three measures of (un)fairness overlap and where they disagree with each other leading to potential tradeoffs. We also include numerical simulations to complement our results.

A Unified View of Group Fairness Tradeoffs Using Partial Information Decomposition

TL;DR

A novel information-theoretic perspective on the relationship between prominent group fairness notions in machine learning, namely statistical parity, equalized odds, and predictive parity is introduced by leveraging a body of work in information theory called partial information decomposition (PID).

Abstract

This paper introduces a novel information-theoretic perspective on the relationship between prominent group fairness notions in machine learning, namely statistical parity, equalized odds, and predictive parity. It is well known that simultaneous satisfiability of these three fairness notions is usually impossible, motivating practitioners to resort to approximate fairness solutions rather than stringent satisfiability of these definitions. However, a comprehensive analysis of their interrelations, particularly when they are not exactly satisfied, remains largely unexplored. Our main contribution lies in elucidating an exact relationship between these three measures of (un)fairness by leveraging a body of work in information theory called partial information decomposition (PID). In this work, we leverage PID to identify the granular regions where these three measures of (un)fairness overlap and where they disagree with each other leading to potential tradeoffs. We also include numerical simulations to complement our results.
Paper Structure (10 sections, 11 theorems, 33 equations, 5 figures, 1 table)

This paper contains 10 sections, 11 theorems, 33 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Background on Partial Information Decomposition
  5. Decomposition of the Measures of Unfairness
  6. Tradeoffs Between Unfairness Measures
  7. Experimental Demonstrations
  8. Conclusion
  9. Proofs for Theorems in Section \ref{['sec:fairdecomp']}
  10. Proofs for Theorems in Section \ref{['sec:tradeoff']}

Key Result

Proposition 1

The statistical parity gap $\mathrm{I}({Z;\hat{Y}})$, equalized odds gap $\mathrm{I}({Z;\hat{Y}|Y})$, and predictive parity gap $\mathrm{I}({Z;Y|\hat{Y}})$ can be decomposed into nonnegative terms as follows:

Figures (5)

  • Figure 1: Illustrates the decomposition of mutual information $\mathrm{I}({Z;\hat{Y},Y})$ using the chain rule. (left) shows the decomposition into Statistical Parity and Predictive Parity. (right) shows the decomposition into $\mathrm{I}({Z;Y})$ and Equalized Odds. No further insights into the overlapping regions of these measures, highlighting the need for measures to capture the nuanced interactions between fairness measures.
  • Figure 2: Venn diagram showing PID of $\mathrm{I}({Z;A,B})$.
  • Figure 3: Blackwell sufficiency of channel $P_{B|Z}$ with respect to $P_{A|Z}$ means $A$ has no unique information about $Z$ that is not in $B$.
  • Figure 4: Venn diagram showing the exact relationship between the various unfairness measures using PID: A critical observation is that all four PID terms are nonnegative. This enables us to derive several fundamental limits and tradeoffs among the unfairness measures, providing a nuanced understanding of when they agree and disagree.
  • Figure 5: (left) Illustrates Theorem \ref{['thm:zero_SP']}, showing that when Statistical Parity is satisfied, the Predictive Parity gap is greater than or equal to the Equalized Odds gap, and if $\mathrm{I}({Z;Y}) {=} 0$, then $\mathrm{I}({Z;Y | \hat{Y}}) {=} \mathrm{I}({Z;\hat{Y} | Y})$. (right) visualizes Theorem \ref{['thm:EOtradeoff']} illustrating that when Equalized Odds is satisfied and $\mathrm{I}({Z;Y}) {>} 0$, there is an inverse relationship (tradeoff) between Statistical Parity and Predictive Parity ($\mathrm{I}({Z;\hat{Y}}) {=} \mathrm{I}({Z;Y}) {-} \mathrm{I}({Z;Y | \hat{Y}})$) since $\mathrm{I}({Z;Y})$ is fixed.

Theorems & Definitions (24)

  • Definition 1: Unique Information bertschinger_QUI
  • Definition 2: Statistical Parity Gap
  • Definition 3: Equalized Odds Gap
  • Definition 4: Predictive Parity Gap
  • Proposition 1
  • Example 1: Pure Uniqueness to Model Prediction
  • Example 2: Pure Redundancy
  • Example 3: Pure Synergy
  • Example 4: Pure Uniqueness to True Label
  • Theorem 1: Revisiting Impossibility
  • ...and 14 more