Demystifying Local and Global Fairness Trade-offs in Federated Learning Using Partial Information Decomposition

TL;DR

This work reframes fairness in federated learning through an information-theoretic lens, defining Global disparity as $I(Z;\hat{Y})$ and Local disparity as $I(Z;\hat{Y}|S)$ to study the interplay under data heterogeneity. Using Partial Information Decomposition, it identifies Unique, Redundant, and Masked disparities that contribute to global and local unfairness, and derives fundamental limits on their trade-offs. The authors introduce the convex Optimization problem AGLFOP to characterize the best achievable accuracy given global/local fairness constraints, and validate the theory with synthetic data and the ADULT dataset, revealing when one fairness notion can or cannot imply the other. The framework offers practical insights for disparity mitigation and auditing in FL, and points to extensions to personalized FL and other fairness notions while noting computational considerations in PID estimation and optimization. Overall, the paper provides a principled, quantitative map of when and how local and global fairness align or conflict in heterogeneous FL environments.

Abstract

This work presents an information-theoretic perspective to group fairness trade-offs in federated learning (FL) with respect to sensitive attributes, such as gender, race, etc. Existing works often focus on either $\textit{global fairness}$ (overall disparity of the model across all clients) or $\textit{local fairness}$ (disparity of the model at each client), without always considering their trade-offs. There is a lack of understanding regarding the interplay between global and local fairness in FL, particularly under data heterogeneity, and if and when one implies the other. To address this gap, we leverage a body of work in information theory called partial information decomposition (PID), which first identifies three sources of unfairness in FL, namely, $\textit{Unique Disparity}$, $\textit{Redundant Disparity}$, and $\textit{Masked Disparity}$. We demonstrate how these three disparities contribute to global and local fairness using canonical examples. This decomposition helps us derive fundamental limits on the trade-off between global and local fairness, highlighting where they agree or disagree. We introduce the $\textit{Accuracy and Global-Local Fairness Optimality Problem (AGLFOP)}$, a convex optimization that defines the theoretical limits of accuracy and fairness trade-offs, identifying the best possible performance any FL strategy can attain given a dataset and client distribution. We also present experimental results on synthetic datasets and the ADULT dataset to support our theoretical findings.

Figures (5)

• Figure 1: Venn diagram showing PID of mutual information $\mathrm{I}({Z;A, B}).$
• Figure 2: Venn diagram of PID for Global & Local Disp. with agreement and disagreement regions.
• Figure 3: AGLFOP Pareto Frontiers for Synthetic and Adult Datasets with PID. (first column) shows maximum accuracy $(1-err)$ that can be achieved on a dataset and client distribution for a given global and local fairness relaxation $(\epsilon_g,\epsilon_l)$. Synthetic data in scenario $1$(first row) is characterized by Unique Disparity. Local and global fairness agree, and accuracy trade-offs are balanced between them. Synthetic data in scenario $2$ with $\alpha=0.9$(second row) is dominated by Redundant Disparity with trade-offs mainly between global fairness and accuracy (an accurate model could have zero Local Disparity but be globally unfair). Synthetic data in Scenario 3 (third row) is characterized by Masked Disparity with trade-offs mainly between local fairness and accuracy (an accurate model could have zero Global Disparity but be locally unfair). Adult data with heterogeneous split (fourth row; details in Appendix \ref{['apx:exp']}), displaying predominantly Masked Disparity but notable presence of Redundant Disparity, capturing more complex relationships and trade-offs.
• Figure 5: Plot demonstrating scenarios with Unique, Redundant, and Masked Disparities for the Adult dataset 5 client case. Difficulty in splitting to achieve pure Redundant and Masked Disparity due to the proportion of labels in the dataset.
• Figure 6: Plot showing the PID of disparities when the data is near i.i.d. among $K=10$ clients. All types of disparities can be observed. The value $\alpha = 0.33$ represents the case where the data is i.i.d. and only Unique Disparity is observed.

Theorems & Definitions (46)

• Definition 1: Unique Information
• Definition 2: Global Disparity
• Lemma 1: Relationship between Global Statistical Parity Gap and $\mut{Z}{\hat{Y}}$
• Definition 3: Local Disparity
• Lemma 2
• Proposition 1
• Example 1: Pure Uniqueness
• Example 2: Pure Redundancy
• Example 3: Pure Synergy
• Theorem 1: Impossibility of Using Local Fairness to Attain Global Fairness