Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Algorithms for Multi-Agent Multi-Armed Bandits with Collision

Daoyuan Zhou, Xuchuang Wang, Lin Yang, Yang Gao

TL;DR

This work proposes a distributed algorithm with an adaptive, efficient communication protocol that achieves near-optimal group and individual regret, with a communication cost of only $\mathcal{O}(\log\log T)$.

Abstract

We study the stochastic Multiplayer Multi-Armed Bandit (MMAB) problem, where multiple players select arms to maximize their cumulative rewards. Collisions occur when two or more players select the same arm, resulting in no reward, and are observed by the players involved. We consider a distributed setting without central coordination, where each player can only observe their own actions and collision feedback. We propose a distributed algorithm with an adaptive, efficient communication protocol. The algorithm achieves near-optimal group and individual regret, with a communication cost of only $\mathcal{O}(\log\log T)$. Our experiments demonstrate significant performance improvements over existing baselines. Compared to state-of-the-art (SOTA) methods, our approach achieves a notable reduction in individual regret. Finally, we extend our approach to a periodic asynchronous setting, proving the lower bound for this problem and presenting an algorithm that achieves logarithmic regret.

Distributed Algorithms for Multi-Agent Multi-Armed Bandits with Collision

TL;DR

This work proposes a distributed algorithm with an adaptive, efficient communication protocol that achieves near-optimal group and individual regret, with a communication cost of only .

Abstract

We study the stochastic Multiplayer Multi-Armed Bandit (MMAB) problem, where multiple players select arms to maximize their cumulative rewards. Collisions occur when two or more players select the same arm, resulting in no reward, and are observed by the players involved. We consider a distributed setting without central coordination, where each player can only observe their own actions and collision feedback. We propose a distributed algorithm with an adaptive, efficient communication protocol. The algorithm achieves near-optimal group and individual regret, with a communication cost of only . Our experiments demonstrate significant performance improvements over existing baselines. Compared to state-of-the-art (SOTA) methods, our approach achieves a notable reduction in individual regret. Finally, we extend our approach to a periodic asynchronous setting, proving the lower bound for this problem and presenting an algorithm that achieves logarithmic regret.

Paper Structure

This paper contains 22 sections, 18 theorems, 108 equations, 1 figure, 1 table, 9 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model
  4. Algorithms
  5. Elimination-Based Learning Policy
  6. Elimination and Acceptance.
  7. Adaptive Communication Protocol
  8. Theoretical Analysis
  9. Extending to Asynchronous Settings
  10. Asynchronous Setups
  11. Algorithm
  12. Theorems
  13. Numerical Results
  14. Conclusion
  15. Supplementary materials for Algorithms
  16. ...and 7 more sections

Key Result

Theorem 1

Set $\delta = 1/T^2$, $\beta > 1$. Then Algorithm alg:SynCD achieves the following performance guarantees, under appropriately calibrated confidence intervals and communication control. where $\Delta(k) = \mu(k) - \mu(M+1)$ if $k \le M$ and $\Delta(k) = \mu(M) - \mu(k)$ if $k > M$ and C denotes the constant regret during initialization.

Figures (1)

  • Figure 1: Comparison between our proposed algorithm and baselines listed in Table \ref{['tab:mmab-comparison']}. Each subplot shows a different performance metric.

Theorems & Definitions (32)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Lemma 4
  • proof
  • Lemma 5
  • proof
  • Lemma 6
  • proof
  • Lemma 7
  • ...and 22 more