Distributed Multi-Agent Bandits Over Erdős-Rényi Random Networks
Jingyuan Liu, Hao Qiu, Lin Yang, Mengfan Xu
TL;DR
This paper addresses distributed MA-MAB with heterogeneous arm rewards over time-varying Erdős–Rényi networks induced by a fixed base graph, where communication edges appear independently with probability $p$. It introduces Gossip Successive Elimination (GSE), a fully distributed algorithm that blends arm-elimination with a refined gossip consensus mechanism, and proves a regret bound of order $O\left(\sum_{k: \Delta_k>0} \frac{\log T}{\Delta_k} + \frac{N^2 \log T}{p \lambda_{N-1}(\operatorname{Lap}(\mathcal{G}))} + \frac{K N^2 \log T}{p}\right)$, illustrating a fundamental trade-off between communication efficiency and learning performance. A matching lower bound for Gaussian instances supports near-optimality of the centralized component, while experimental results on synthetic and real data validate the theoretical scaling and show the impact of $p$ and base-graph connectivity. The work generalizes previous MA-MAB results to general ER graphs induced by a base graph, highlights the influence of topology on regret, and provides practical guidance for balancing communication cost against regret in decentralized learning systems.
Abstract
We study the distributed multi-agent multi-armed bandit problem with heterogeneous rewards over random communication graphs. Uniquely, at each time step $t$ agents communicate over a time-varying random graph $G_t$ generated by applying the Erdős-Rényi model to a fixed connected base graph $G$ (for classical Erdős-Rényi graphs, $G$ is a complete graph), where each potential edge in $G$ is randomly and independently present with the link probability $p$. Notably, the resulting random graph is not necessarily connected at each time step. Each agent's arm rewards follow time-invariant distributions, and the reward distribution for the same arm may differ across agents. The goal is to minimize the cumulative expected regret relative to the global mean reward of each arm, defined as the average of that arm's mean rewards across all agents. To this end, we propose a fully distributed algorithm that integrates the arm elimination strategy with the random gossip algorithm. We theoretically show that the regret upper bound is of order $\log T$ and is highly interpretable, where $T$ is the time horizon. It includes the optimal centralized regret $O\left(\sum_{k: Δ_k>0} \frac{\log T}{Δ_k}\right)$ and an additional term $O\left(\frac{N^2 \log T}{p λ_{N-1}(Lap(G))} + \frac{KN^2 \log T}{p}\right)$ where $N$ and $K$ denote the total number of agents and arms, respectively. This term reflects the impact of $G$'s algebraic connectivity $λ_{N-1}(Lap(G))$ and the link probability $p$, and thus highlights a fundamental trade-off between communication efficiency and regret. As a by-product, we show a nearly optimal regret lower bound. Finally, our numerical experiments not only show the superiority of our algorithm over existing benchmarks, but also validate the theoretical regret scaling with problem complexity.