Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Understanding Byzantine Robustness in Federated Learning with A Black-box Server

Fangyuan Zhao, Yuexiang Xie, Xuebin Ren, Bolin Ding, Shusen Yang, Yaliang Li

TL;DR

This paper studies Byzantine robustness in federated learning with a black-box server. It introduces dynamic defense strategies that randomly select among a set of robust aggregation rules, quantified by a probability distribution, to hinder adversaries, and provides theoretical results showing robustness and convergence properties under these dynamic defenses. The authors prove that a black-box server can reduce the worst-case attack impact from a maximally unfavorable level to an expected level, due to the inaccessibility of the defense strategy to attackers, and they validate these claims with extensive experiments on FEMNIST and CIFAR-10 against both AGR-agnostic and AGR-adaptive attacks. The work highlights the practical benefits of incorporating randomness and inaccessibility into aggregation, and releases code to support further research in Byzantine-robust federated learning.

Abstract

Federated learning (FL) becomes vulnerable to Byzantine attacks where some of participators tend to damage the utility or discourage the convergence of the learned model via sending their malicious model updates. Previous works propose to apply robust rules to aggregate updates from participators against different types of Byzantine attacks, while at the same time, attackers can further design advanced Byzantine attack algorithms targeting specific aggregation rule when it is known. In practice, FL systems can involve a black-box server that makes the adopted aggregation rule inaccessible to participants, which can naturally defend or weaken some Byzantine attacks. In this paper, we provide an in-depth understanding on the Byzantine robustness of the FL system with a black-box server. Our investigation demonstrates the improved Byzantine robustness of a black-box server employing a dynamic defense strategy. We provide both empirical evidence and theoretical analysis to reveal that the black-box server can mitigate the worst-case attack impact from a maximum level to an expectation level, which is attributed to the inherent inaccessibility and randomness offered by a black-box server.The source code is available at https://github.com/alibaba/FederatedScope/tree/Byzantine_attack_defense to promote further research in the community.

Understanding Byzantine Robustness in Federated Learning with A Black-box Server

TL;DR

This paper studies Byzantine robustness in federated learning with a black-box server. It introduces dynamic defense strategies that randomly select among a set of robust aggregation rules, quantified by a probability distribution, to hinder adversaries, and provides theoretical results showing robustness and convergence properties under these dynamic defenses. The authors prove that a black-box server can reduce the worst-case attack impact from a maximally unfavorable level to an expected level, due to the inaccessibility of the defense strategy to attackers, and they validate these claims with extensive experiments on FEMNIST and CIFAR-10 against both AGR-agnostic and AGR-adaptive attacks. The work highlights the practical benefits of incorporating randomness and inaccessibility into aggregation, and releases code to support further research in Byzantine-robust federated learning.

Abstract

Federated learning (FL) becomes vulnerable to Byzantine attacks where some of participators tend to damage the utility or discourage the convergence of the learned model via sending their malicious model updates. Previous works propose to apply robust rules to aggregate updates from participators against different types of Byzantine attacks, while at the same time, attackers can further design advanced Byzantine attack algorithms targeting specific aggregation rule when it is known. In practice, FL systems can involve a black-box server that makes the adopted aggregation rule inaccessible to participants, which can naturally defend or weaken some Byzantine attacks. In this paper, we provide an in-depth understanding on the Byzantine robustness of the FL system with a black-box server. Our investigation demonstrates the improved Byzantine robustness of a black-box server employing a dynamic defense strategy. We provide both empirical evidence and theoretical analysis to reveal that the black-box server can mitigate the worst-case attack impact from a maximum level to an expectation level, which is attributed to the inherent inaccessibility and randomness offered by a black-box server.The source code is available at https://github.com/alibaba/FederatedScope/tree/Byzantine_attack_defense to promote further research in the community.
Paper Structure (33 sections, 2 theorems, 29 equations, 6 figures, 1 algorithm)

This paper contains 33 sections, 2 theorems, 29 equations, 6 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Federated Learning
  4. Byzantine Robustness
  5. Byzantine Attacks in FL
  6. Assumptions
  7. Byzantine Robustness in FL with A Black-box Server
  8. Attack and Defense Model
  9. Static Defense Strategies
  10. Dynamic Defense Strategies
  11. White-box Servers with Dynamic Defense v.s. Black-box Servers with Dynamic Defense
  12. Experiments
  13. Experimental Settings
  14. Datasets and Models
  15. Aggregation Rules and Attack Algorithms
  16. ...and 18 more sections

Key Result

Theorem 1

Let $B = \{B_1, \ldots, B_h\}$ be an attack which can successfully attack a subset $\mathcal{S}^{\prime}$ of $q$ aggregation rules in $\mathcal{S}$, W.L.O.G., $\mathcal{S}^{\prime} = \{\mathcal{AGR}_j, j\in[q]\}$, in the sense that $\forall \mathcal{AGR}_j \in \mathcal{S}^{\prime}$, it holds that $\ where $Q_i = \mathcal{AGR}_i(V, B)$, and the robust level is $(h, \mathbb{E}_{\mathcal{AGR}_{i}\sim

Figures (6)

  • Figure 1: Defending against AGR-agnostic attacks on FEMNIST.
  • Figure 2: Defending against AGR-adaptive attacks on FEMNIST.
  • Figure 3: Defending against AGR-adaptive attacks on CIFAR-10.
  • Figure 4: Negative impact caused by AGR-adaptive attacks w.r.t. the number of clients.
  • Figure 5: Negative impact caused by AGR-adaptive attacks w.r.t. the degree of data heterogeneity.
  • ...and 1 more figures

Theorems & Definitions (4)

  • Theorem 1: Robustness Analysis
  • Theorem 2: Convergence Analysis
  • Proof B.1
  • Proof B.2