Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Combinatorial Multi-Agent Multi-Armed Bandits

Fares Fourati, Mohamed-Slim Alouini, Vaneet Aggarwal

TL;DR

This work tackles online stochastic combinatorial optimization under bandit feedback in a federated, partially participatory setting. It introduces C-MA-MAB, a general offline-to-online framework that leverages any resilient offline $(\alpha-\epsilon)$-approximation as a black-box subroutine to achieve sublinear $\alpha$-regret with a linear speedup as the number of communicating agents grows, while requiring only sublinear communication rounds. Theoretical results provide explicit regret and communication bounds parameterized by $(\alpha,\beta,\gamma,\psi,\delta)$ and show applicability to online submodular maximization, including both monotone and non-monotone cases, plus knapsack variants. Empirical validation on stochastic data summarization tasks demonstrates the framework’s effectiveness and its advantage over baselines, validating both the practicality and scalability of federated combinatorial bandits in real-world settings.

Abstract

This paper introduces a federated learning framework tailored for online combinatorial optimization with bandit feedback. In this setting, agents select subsets of arms, observe noisy rewards for these subsets without accessing individual arm information, and can cooperate and share information at specific intervals. Our framework transforms any offline resilient single-agent $(α-ε)$-approximation algorithm, having a complexity of $\tilde{\mathcal{O}}(\fracψ{ε^β})$, where the logarithm is omitted, for some function $ψ$ and constant $β$, into an online multi-agent algorithm with $m$ communicating agents and an $α$-regret of no more than $\tilde{\mathcal{O}}(m^{-\frac{1}{3+β}} ψ^\frac{1}{3+β} T^\frac{2+β}{3+β})$. This approach not only eliminates the $ε$ approximation error but also ensures sublinear growth with respect to the time horizon $T$ and demonstrates a linear speedup with an increasing number of communicating agents. Additionally, the algorithm is notably communication-efficient, requiring only a sublinear number of communication rounds, quantified as $\tilde{\mathcal{O}}\left(ψT^\fracβ{β+1}\right)$. Furthermore, the framework has been successfully applied to online stochastic submodular maximization using various offline algorithms, yielding the first results for both single-agent and multi-agent settings and recovering specialized single-agent theoretical guarantees. We empirically validate our approach to a stochastic data summarization problem, illustrating the effectiveness of the proposed framework, even in single-agent scenarios.

Federated Combinatorial Multi-Agent Multi-Armed Bandits

TL;DR

This work tackles online stochastic combinatorial optimization under bandit feedback in a federated, partially participatory setting. It introduces C-MA-MAB, a general offline-to-online framework that leverages any resilient offline -approximation as a black-box subroutine to achieve sublinear -regret with a linear speedup as the number of communicating agents grows, while requiring only sublinear communication rounds. Theoretical results provide explicit regret and communication bounds parameterized by and show applicability to online submodular maximization, including both monotone and non-monotone cases, plus knapsack variants. Empirical validation on stochastic data summarization tasks demonstrates the framework’s effectiveness and its advantage over baselines, validating both the practicality and scalability of federated combinatorial bandits in real-world settings.

Abstract

This paper introduces a federated learning framework tailored for online combinatorial optimization with bandit feedback. In this setting, agents select subsets of arms, observe noisy rewards for these subsets without accessing individual arm information, and can cooperate and share information at specific intervals. Our framework transforms any offline resilient single-agent -approximation algorithm, having a complexity of , where the logarithm is omitted, for some function and constant , into an online multi-agent algorithm with communicating agents and an -regret of no more than . This approach not only eliminates the approximation error but also ensures sublinear growth with respect to the time horizon and demonstrates a linear speedup with an increasing number of communicating agents. Additionally, the algorithm is notably communication-efficient, requiring only a sublinear number of communication rounds, quantified as . Furthermore, the framework has been successfully applied to online stochastic submodular maximization using various offline algorithms, yielding the first results for both single-agent and multi-agent settings and recovering specialized single-agent theoretical guarantees. We empirically validate our approach to a stochastic data summarization problem, illustrating the effectiveness of the proposed framework, even in single-agent scenarios.
Paper Structure (34 sections, 18 theorems, 67 equations, 3 figures, 2 tables, 1 algorithm)

This paper contains 34 sections, 18 theorems, 67 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Related Work
  4. Combinatorial MA-MAB Framework
  5. Offline Resilient-Approximation
  6. Offline-to-Online Multi-Agent Framework
  7. Theoretical Analysis
  8. Communication Analysis
  9. Estimation Analysis
  10. Regret Analysis
  11. regret of an Agent under the Good Event
  12. Regret of an Agent under Any Event
  13. Application to Submodular Maximization
  14. Experiments with Data Summarization
  15. Notation
  16. ...and 19 more sections

Key Result

Lemma 5.1

The number of communication times, i.e., the number of oracle queries $N(\beta, \gamma, \psi, \epsilon^\star)$ of the subroutine $(\alpha, \beta, \gamma, \psi, \delta)$-resilient-approximation algorithm $\mathcal{A}(\epsilon^\star)$ satisfies: $N(\beta, \gamma, \psi, \epsilon^\star) \leq \mathc

Figures (3)

  • Figure 1: Cumulative regrets of summarizing images from CIFAR10 for different horizons $T$ using our C-MA-MAB framework with different number of agents $m$, against C-ETC-N and UCB1.
  • Figure 2: Cumulative regrets of summarizing images from FMNIST for different time horizons $T$ using our framework with different number of agents against C-ETC framework and UCB.
  • Figure 3: Compare the instantaneous rewards of summarizing images from CIFAR10 and FMNIST for different time horizons $T$ using our framework with one, two, four, sixteen, and thirtytwo agents against C-ETC-N, UCB1, and OPT.

Theorems & Definitions (38)

  • Definition 4.1: $\xi$-controlled-estimation
  • Definition 4.2: $(\alpha, \beta, \gamma, \psi, \delta)$-resilient approximation
  • Remark 4.3
  • Remark 4.4
  • Remark 4.5
  • Remark 4.6
  • Remark 4.7
  • Lemma 5.1
  • Lemma 5.2
  • Theorem 5.3
  • ...and 28 more