Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning with Relative Fairness

Shogo Nakakita, Tatsuya Kaneko, Shinya Takamaeda-Yamazaki, Masaaki Imaizumi

TL;DR

The proposed framework uses a minimax problem approach to minimize relative unfairness, extending previous methods in distributionally robust optimization (DRO) and introducing a novel fairness index, based on the ratio between large and small losses among clients, allowing the framework to assess and improve the relative fairness of trained models.

Abstract

This paper proposes a federated learning framework designed to achieve \textit{relative fairness} for clients. Traditional federated learning frameworks typically ensure absolute fairness by guaranteeing minimum performance across all client subgroups. However, this approach overlooks disparities in model performance between subgroups. The proposed framework uses a minimax problem approach to minimize relative unfairness, extending previous methods in distributionally robust optimization (DRO). A novel fairness index, based on the ratio between large and small losses among clients, is introduced, allowing the framework to assess and improve the relative fairness of trained models. Theoretical guarantees demonstrate that the framework consistently reduces unfairness. We also develop an algorithm, named \textsc{Scaff-PD-IA}, which balances communication and computational efficiency while maintaining minimax-optimal convergence rates. Empirical evaluations on real-world datasets confirm its effectiveness in maintaining model performance while reducing disparity.

Federated Learning with Relative Fairness

TL;DR

The proposed framework uses a minimax problem approach to minimize relative unfairness, extending previous methods in distributionally robust optimization (DRO) and introducing a novel fairness index, based on the ratio between large and small losses among clients, allowing the framework to assess and improve the relative fairness of trained models.

Abstract

This paper proposes a federated learning framework designed to achieve \textit{relative fairness} for clients. Traditional federated learning frameworks typically ensure absolute fairness by guaranteeing minimum performance across all client subgroups. However, this approach overlooks disparities in model performance between subgroups. The proposed framework uses a minimax problem approach to minimize relative unfairness, extending previous methods in distributionally robust optimization (DRO). A novel fairness index, based on the ratio between large and small losses among clients, is introduced, allowing the framework to assess and improve the relative fairness of trained models. Theoretical guarantees demonstrate that the framework consistently reduces unfairness. We also develop an algorithm, named \textsc{Scaff-PD-IA}, which balances communication and computational efficiency while maintaining minimax-optimal convergence rates. Empirical evaluations on real-world datasets confirm its effectiveness in maintaining model performance while reducing disparity.

Paper Structure

This paper contains 47 sections, 21 theorems, 140 equations, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Notation
  4. Preliminary
  5. Distribution Robust Optimization for Fair Federated Learning
  6. Learning Algorithm for DRO
  7. Proposal: Learning Framework with Relative (Un)Fairness
  8. Relative Unfairness Index
  9. Relative Fairness Discrepancy by Majorization Approximation
  10. Proposed Framework
  11. Theory on Fairness Improvement
  12. Weak Improvement
  13. Strong Improvement
  14. Simulation Study
  15. Algorithm
  16. ...and 32 more sections

Key Result

Theorem 1

Under Assumption assump:nozerodiv, $\mathfrak{R}_{A,B}(\theta_{\varphi}^{\star})$ is non-increasing in $\varphi\in[0,1)$, that is, we have $\mathfrak{R}_{A,B}(\theta_{\varphi}^{\star})\le \mathfrak{R}_{A,B}(\theta_{0}^{\star})$ for any $\varphi\in (0,1)$.

Figures (4)

  • Figure 1: Illustration of absolute fairness and relative fairness. In the left panel, distances from the eight gray points to the pink/blue center point are measured, respectively, and the right figure shows the distribution of those distances. While the pink distribution achieves absolute fairness which minimizes of the maximum distance, the blue distribution achieves relative fairness which minimizes the discrepancy among the distances. Section \ref{['sec:figure_explain']} shows more details.
  • Figure 2: Plots of $\Lambda_{\varphi}(\Delta_{0.5}^{2},\Delta_{0.5}^{2})$, with $\varphi= 0$ (pink), $\varphi = 0.2$ (orange), and $\varphi = 0.5$ (green). The set with solid lines is a regular ambiguity set ($\Lambda_{0}(\Delta_{0.5}^{2},\Delta_{0.5}^{2})=\Delta_{0.5}^{2}$), and with dashed lines is an integrated set.
  • Figure 3: Relative unfairness indices in linear regression for the penguins data with three clients defined by Adelie, Chinstrap, and Gentoo.
  • Figure 4: (Transformed) Lorenz curves of losses in the MNIST experiment. The smaller the area above the curve, the smaller unfairness it achieves.

Theorems & Definitions (44)

  • Example 1: Value-at-Risk
  • Definition 1: Relative unfairness index
  • Example 2: Relative fairness index with VaR
  • Definition 2: Relative fairness discrepancy
  • Theorem 1
  • Theorem 2
  • Theorem 3: Convergence of Scaff-PD-IA
  • Theorem 4: bounds for generalization error
  • Proposition 5
  • Lemma 6
  • ...and 34 more