Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Decentralized Multi-armed Bandits: From Corruption-Resilience to Byzantine-Resilience

Zicheng Hu, Yuchen Wang, Cheng Chen

TL;DR

This work tackles robust coordination in decentralized multi-armed bandits by addressing both adversarial reward corruption and Byzantine agents. The authors introduce DeMABAR, an epoch-based algorithm that combines restricted, neighbor-based collaboration with a novel filtering mechanism to mitigate corrupted data, achieving near-optimal per-agent regret with only O(wV ln(VT)) communication. Theoretical results show that DeMABAR maintains strong regret guarantees under adversarial corruption with β ≤ α, and remains robust in Byzantine settings with an unknown fraction α of Byzantine neighbors, incurring only additive terms tied to the total corruption budget C. Empirical results corroborate the theory, demonstrating substantial performance gains over baselines in centralized, decentralized, and Byzantine scenarios. The work advances practical, robust cooperative learning for networked agents, enabling reliable decision-making in adversarial environments while keeping communication overhead manageable.

Abstract

Decentralized cooperative multi-agent multi-armed bandits (DeCMA2B) considers how multiple agents collaborate in a decentralized multi-armed bandit setting. Though this problem has been extensively studied in previous work, most existing methods remain susceptible to various adversarial attacks. In this paper, we first study DeCMA2B with adversarial corruption, where an adversary can corrupt reward observations of all agents with a limited corruption budget. We propose a robust algorithm, called DeMABAR, which ensures that each agent's individual regret suffers only an additive term proportional to the corruption budget. Then we consider a more realistic scenario where the adversary can only attack a small number of agents. Our theoretical analysis shows that the DeMABAR algorithm can also almost completely eliminate the influence of adversarial attacks and is inherently robust in the Byzantine setting, where an unknown fraction of the agents can be Byzantine, i.e., may arbitrarily select arms and communicate wrong information. We also conduct numerical experiments to illustrate the robustness and effectiveness of the proposed method.

Robust Decentralized Multi-armed Bandits: From Corruption-Resilience to Byzantine-Resilience

TL;DR

This work tackles robust coordination in decentralized multi-armed bandits by addressing both adversarial reward corruption and Byzantine agents. The authors introduce DeMABAR, an epoch-based algorithm that combines restricted, neighbor-based collaboration with a novel filtering mechanism to mitigate corrupted data, achieving near-optimal per-agent regret with only O(wV ln(VT)) communication. Theoretical results show that DeMABAR maintains strong regret guarantees under adversarial corruption with β ≤ α, and remains robust in Byzantine settings with an unknown fraction α of Byzantine neighbors, incurring only additive terms tied to the total corruption budget C. Empirical results corroborate the theory, demonstrating substantial performance gains over baselines in centralized, decentralized, and Byzantine scenarios. The work advances practical, robust cooperative learning for networked agents, enabling reliable decision-making in adversarial environments while keeping communication overhead manageable.

Abstract

Decentralized cooperative multi-agent multi-armed bandits (DeCMA2B) considers how multiple agents collaborate in a decentralized multi-armed bandit setting. Though this problem has been extensively studied in previous work, most existing methods remain susceptible to various adversarial attacks. In this paper, we first study DeCMA2B with adversarial corruption, where an adversary can corrupt reward observations of all agents with a limited corruption budget. We propose a robust algorithm, called DeMABAR, which ensures that each agent's individual regret suffers only an additive term proportional to the corruption budget. Then we consider a more realistic scenario where the adversary can only attack a small number of agents. Our theoretical analysis shows that the DeMABAR algorithm can also almost completely eliminate the influence of adversarial attacks and is inherently robust in the Byzantine setting, where an unknown fraction of the agents can be Byzantine, i.e., may arbitrarily select arms and communicate wrong information. We also conduct numerical experiments to illustrate the robustness and effectiveness of the proposed method.

Paper Structure

This paper contains 39 sections, 18 theorems, 119 equations, 4 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. DeCMA2B.
  4. DeCMA2B with Adversarial Corruptions.
  5. DeCMA2B with Byzantine Agents.
  6. Preliminaries
  7. Problem Setup
  8. Multi-agent Multi-armed Bandits
  9. Adversarially corrupted setting
  10. Byzantine setting
  11. Relationship between adversarially corrupted and Byzantine settings
  12. Notation
  13. Algorithm
  14. DeCMA2B with Adversarial Corruptions
  15. Filtering mechanism
  16. ...and 24 more sections

Key Result

Theorem 1

In DeCMA2B with adversarial corruptions, our DeMABAR algorithm only requires a communication cost of $O(wV \ln(VT))$ to achieve the following individual regret for each agent $i$: If $\beta \le \alpha$, we have If $\beta > \alpha$, we have

Figures (4)

  • Figure 1: DeMABAR vs. DRAA, Resilient Decentralized UCB, MA-BARBAT, IND-BARBAR, and IND-FTRL in centralized CMA2B under adversarial corruption.
  • Figure 2: The network structure used in the experiment.
  • Figure 3: DeMABAR vs. Resilient Decentralized UCB, IND-BARBAR, and IND-FTRL in DeCMA2B under adversarial corruption.
  • Figure 4: DeMABAR-F vs. Resilient Decentralized UCB, IND-BARBAR, and IND-FTRL in DeCMA2B with Byzantine agents. The Byzantine agents are the black nodes in Figure \ref{['fig:gra']}.

Theorems & Definitions (35)

  • Theorem 1
  • Remark 1
  • Corollary 1
  • Remark 2
  • Theorem 2
  • Remark 3
  • Remark 4
  • Lemma 1
  • proof
  • Lemma 2
  • ...and 25 more