Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedMABA: Towards Fair Federated Learning through Multi-Armed Bandits Allocation

Zhichao Wang, Lin Wang, Yongxin Guo, Ying-Jun Angela Zhang, Xiaoying Tang

TL;DR

This paper proposes a novel multiarmed bandit-based allocation FL algorithm (FedMABA) to mitigate performance unfairness among diverse clients with different data distributions and introduces the concept of adversarial multi-armed bandit to optimize the proposed objective with explicit constraints on performance disparities.

Abstract

The increasing concern for data privacy has driven the rapid development of federated learning (FL), a privacy-preserving collaborative paradigm. However, the statistical heterogeneity among clients in FL results in inconsistent performance of the server model across various clients. Server model may show favoritism towards certain clients while performing poorly for others, heightening the challenge of fairness. In this paper, we reconsider the inconsistency in client performance distribution and introduce the concept of adversarial multi-armed bandit to optimize the proposed objective with explicit constraints on performance disparities. Practically, we propose a novel multi-armed bandit-based allocation FL algorithm (FedMABA) to mitigate performance unfairness among diverse clients with different data distributions. Extensive experiments, in different Non-I.I.D. scenarios, demonstrate the exceptional performance of FedMABA in enhancing fairness.

FedMABA: Towards Fair Federated Learning through Multi-Armed Bandits Allocation

TL;DR

This paper proposes a novel multiarmed bandit-based allocation FL algorithm (FedMABA) to mitigate performance unfairness among diverse clients with different data distributions and introduces the concept of adversarial multi-armed bandit to optimize the proposed objective with explicit constraints on performance disparities.

Abstract

The increasing concern for data privacy has driven the rapid development of federated learning (FL), a privacy-preserving collaborative paradigm. However, the statistical heterogeneity among clients in FL results in inconsistent performance of the server model across various clients. Server model may show favoritism towards certain clients while performing poorly for others, heightening the challenge of fairness. In this paper, we reconsider the inconsistency in client performance distribution and introduce the concept of adversarial multi-armed bandit to optimize the proposed objective with explicit constraints on performance disparities. Practically, we propose a novel multi-armed bandit-based allocation FL algorithm (FedMABA) to mitigate performance unfairness among diverse clients with different data distributions. Extensive experiments, in different Non-I.I.D. scenarios, demonstrate the exceptional performance of FedMABA in enhancing fairness.

Paper Structure

This paper contains 14 sections, 3 theorems, 12 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries and Metrics
  4. Basic Federated Learning
  5. Fairness via Variance
  6. Our Fair FL Optimization Objective
  7. FedMABA: MAB Allocation and Aggregation Method
  8. Higher Generalization performance via Lower Performance Disparities
  9. Fair Allocation and Aggregation
  10. Convergence Analysis of FedMABA
  11. Experiments
  12. Baselines, Datasets, Models, and Hyper-parameters
  13. Numerical Results
  14. Conclusion and Future Works

Key Result

Theorem 1

Consider a FL system with N clients, and parameter hypothesis space $\mathcal{W}$. If Assumption bounded loss holds. Then, for $\forall \delta \in \left[0, 1\right]$ with probability of at least $1-2\delta$, for $\forall w \in \mathcal{W}$, the generalization error can be bounded as:

Figures (2)

  • Figure 1: Fairness Performance of Algorithms.
  • Figure 2: Performance of variance on FedMABA. Ablation Study of $\eta_b$ and $\alpha$. Left: Fixing $\alpha = 0.5$ and examining fairness performance for different $\eta_b$ values. Right: Fixing $\eta_b = 0.5$ and evaluating fairness performance for different $\alpha$ values.

Theorems & Definitions (8)

  • Definition 1: Fairness via Variance
  • Remark 1: The difficulty of solving problem \ref{['fair aggregation formulation']}
  • Definition 2: Rademacher Complexity
  • Definition 3: Representation of Generalization Error
  • Theorem 1: Bounded Generalization Error
  • Corollary 1: Bounded Generalization Error
  • Remark 2
  • Theorem 2: Convergence Analysis of FedMABA