Near Optimal Best Arm Identification for Clustered Bandits

TL;DR

The paper tackles best-arm identification in a heterogeneous, multi-agent MAB setting by introducing two phase-based algorithms, Cl-BAI and BAI-Cl, with δ-PC guarantees and near-optimal sample/communication trade-offs. Cl-BAI clusters agents via successive elimination and then identifies cluster-wise best arms, while BAI-Cl first discovers the M distinct best arms and then assigns them to remaining agents, both supported by rigorous bounds and a matching lower bound. An enhanced variant, BAI-Cl++, leverages stronger separability to reduce sample complexity further, achieving order-wise minimax optimality for constant M. Empirical results on synthetic data and real datasets (MovieLens,Yelp) demonstrate substantial sample- and communication-efficiency gains when M ≪ N, validating the theoretical findings and highlighting practical impact for distributed recommendation and federated learning systems.

Abstract

This work investigates the problem of best arm identification for multi-agent multi-armed bandits. We consider $N$ agents grouped into $M$ clusters, where each cluster solves a stochastic bandit problem. The mapping between agents and bandits is a priori unknown. Each bandit is associated with $K$ arms, and the goal is to identify the best arm for each agent under a $δ$-probably correct ($δ$-PC) framework, while minimizing sample complexity and communication overhead. We propose two novel algorithms: Clustering then Best Arm Identification (Cl-BAI) and Best Arm Identification then Clustering (BAI-Cl). Cl-BAI uses a two-phase approach that first clusters agents based on the bandit problems they are learning, followed by identifying the best arm for each cluster. BAI-Cl reverses the sequence by identifying the best arms first and then clustering agents accordingly. Both algorithms leverage the successive elimination framework to ensure computational efficiency and high accuracy. We establish $δ$-PC guarantees for both methods, derive bounds on their sample complexity, and provide a lower bound for this problem class. Moreover, when $M$ is small (a constant), we show that the sample complexity of a variant of BAI-Cl is minimax optimal in an order-wise sense. Experiments on synthetic and real-world datasets (MovieLens, Yelp) demonstrate the superior performance of the proposed algorithms in terms of sample and communication efficiency, particularly in settings where $M \ll N$.

Figures (2)

• Figure 1: Performance with varying number of agents $N$ for different experimental setups. (a) Small dataset. (b) Setup 2, varying $N$. (c) Setup 1, varying $N$. (d) Setup 1, varying $D$. (e) Setup 1, skewness. (f) MovieLens-derived setup.
• Figure 2: (a) Performance with varying number of agents $N$ for Yelp dataset. (b)(c) Communication cost with varying number of agents $N$ for datasets 2, 3.

Theorems & Definitions (62)

• Remark 4.1: Successive Elimination
• Remark 4.2: Knowledge of separation $\eta$
• Theorem 4.3
• Theorem 4.4
• Remark 4.5: Comparison with a naive algorithm
• Remark 4.6: Communication Cost
• Remark 5.1: Coupon Collector
• Theorem 5.2
• Theorem 5.3
• Remark 5.4: Comparison between BAI-Cl and Cl-BAI