Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Multi-Armed Bandits Under Byzantine Attacks

Artun Saday, İlker Demirel, Yiğit Yıldırım, Cem Tekin

TL;DR

This work addresses robust decision-making in federated, non-i.i.d. multi-armed bandits under Byzantine attacks. It introduces Fed-MoM-UCB, a median-of-means based aggregator that tolerates adversarial client updates and achieves $β$-PAC arm identification with high probability while bounding cumulative regret, even when Byzantine clients constitute up to just under half of the cohort. Theoretical results provide MoM concentration bounds, sample complexity, and regret guarantees, highlighting the trade-offs between the discernibility margin, phase lengths, and communication costs. Empirical results across three scenarios validate the method's resilience and efficiency, showing superiority over baselines like Fed2-UCB and Fed-Trim-UCB, and demonstrating practical robustness for secure, scalable federated decision making.

Abstract

Multi-armed bandits (MAB) is a sequential decision-making model in which the learner controls the trade-off between exploration and exploitation to maximize its cumulative reward. Federated multi-armed bandits (FMAB) is an emerging framework where a cohort of learners with heterogeneous local models play an MAB game and communicate their aggregated feedback to a server to learn a globally optimal arm. Two key hurdles in FMAB are communication-efficient learning and resilience to adversarial attacks. To address these issues, we study the FMAB problem in the presence of Byzantine clients who can send false model updates threatening the learning process. We analyze the sample complexity and the regret of $β$-optimal arm identification. We borrow tools from robust statistics and propose a median-of-means (MoM)-based online algorithm, Fed-MoM-UCB, to cope with Byzantine clients. In particular, we show that if the Byzantine clients constitute less than half of the cohort, the cumulative regret with respect to $β$-optimal arms is bounded over time with high probability, showcasing both communication efficiency and Byzantine resilience. We analyze the interplay between the algorithm parameters, a discernibility margin, regret, communication cost, and the arms' suboptimality gaps. We demonstrate Fed-MoM-UCB's effectiveness against the baselines in the presence of Byzantine attacks via experiments.

Federated Multi-Armed Bandits Under Byzantine Attacks

TL;DR

This work addresses robust decision-making in federated, non-i.i.d. multi-armed bandits under Byzantine attacks. It introduces Fed-MoM-UCB, a median-of-means based aggregator that tolerates adversarial client updates and achieves -PAC arm identification with high probability while bounding cumulative regret, even when Byzantine clients constitute up to just under half of the cohort. Theoretical results provide MoM concentration bounds, sample complexity, and regret guarantees, highlighting the trade-offs between the discernibility margin, phase lengths, and communication costs. Empirical results across three scenarios validate the method's resilience and efficiency, showing superiority over baselines like Fed2-UCB and Fed-Trim-UCB, and demonstrating practical robustness for secure, scalable federated decision making.

Abstract

Multi-armed bandits (MAB) is a sequential decision-making model in which the learner controls the trade-off between exploration and exploitation to maximize its cumulative reward. Federated multi-armed bandits (FMAB) is an emerging framework where a cohort of learners with heterogeneous local models play an MAB game and communicate their aggregated feedback to a server to learn a globally optimal arm. Two key hurdles in FMAB are communication-efficient learning and resilience to adversarial attacks. To address these issues, we study the FMAB problem in the presence of Byzantine clients who can send false model updates threatening the learning process. We analyze the sample complexity and the regret of -optimal arm identification. We borrow tools from robust statistics and propose a median-of-means (MoM)-based online algorithm, Fed-MoM-UCB, to cope with Byzantine clients. In particular, we show that if the Byzantine clients constitute less than half of the cohort, the cumulative regret with respect to -optimal arms is bounded over time with high probability, showcasing both communication efficiency and Byzantine resilience. We analyze the interplay between the algorithm parameters, a discernibility margin, regret, communication cost, and the arms' suboptimality gaps. We demonstrate Fed-MoM-UCB's effectiveness against the baselines in the presence of Byzantine attacks via experiments.
Paper Structure (20 sections, 11 theorems, 41 equations, 5 figures, 4 tables, 1 algorithm)

This paper contains 20 sections, 11 theorems, 41 equations, 5 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Learning
  4. Federated Multi-armed Bandits
  5. Motivating Example
  6. Problem Formulation
  7. Federated MAB Setup
  8. Byzantine Clients
  9. Regret
  10. Fed-MoM-UCB Algorithm
  11. Client Side
  12. Server Side
  13. Theoretical Analysis
  14. Preliminary Technical Results
  15. $(\beta,\delta)$-PACness, Sample Complexity and Regret Analysis
  16. ...and 5 more sections

Key Result

Lemma 1

(MoM concentration bound). Fix $k \in [K]$. Let $B=\lfloor1/\alpha(\lambda)\rfloor$, $G = M/B$, $\eta(\lambda) \coloneqq (\alpha(\lambda) - \lambda)/\alpha(\lambda)$, and $\rho := \sigma/\sqrt{s(p)} + \sigma_c$. Then, for all

Figures (5)

  • Figure 1: Illustration of the motivating example.
  • Figure 2: Ten clients where two are Byzantine (red lines), and they perform outlier attacks. Fed-MoM-UCB first divides the clients into five groups. Two example groupings are shown on the right. In the worst case, the Byzantine clients are assigned to different groups and saturate their groups' local updates. The saturation will not affect the median estimate.
  • Figure 3: Regrets and communication costs for Scenario 1.
  • Figure 4: Regrets and communication costs for Scenario 2.
  • Figure 5: Regrets and confidence bounds for Scenario 3.

Theorems & Definitions (25)

  • Definition 1
  • Remark 1
  • Lemma 1
  • Remark 2
  • Remark 3
  • Lemma 2
  • proof
  • Lemma 3
  • Remark 4
  • Lemma 4
  • ...and 15 more