Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Clustering in Stochastic Bandits

G Dhinesh Chandran, Kota Srinivas Reddy, Srikrishna Bhashyam

TL;DR

This work studies Bandit Clustering (BC) under fixed-confidence for arms with vector-parametric distributions, seeking to recover clustering with minimal samples. It introduces Efficient Bandit Clustering (EBC), which uses a one-step gradient toward the optimal sampling proportions instead of full optimization, and a heuristic EBC-H for further simplification. EBC is shown to be $\delta$-PC and asymptotically optimal, with strong computational advantages over prior methods; EBC-H matches the asymptotic lower bound slope and often improves practical performance. Empirical results on synthetic and real-world datasets demonstrate that EBC and EBC-H outperform existing approaches in both sample complexity and runtime. Overall, the methods extend BC to broader distribution families and offer scalable, provably efficient clustering in sequential bandit settings.

Abstract

We study the Bandit Clustering (BC) problem under the fixed confidence setting, where the objective is to group a collection of data sequences (arms) into clusters through sequential sampling from adaptively selected arms at each time step while ensuring a fixed error probability at the stopping time. We consider a setting where arms in a cluster may have different distributions. Unlike existing results in this setting, which assume Gaussian-distributed arms, we study a broader class of vector-parametric distributions that satisfy mild regularity conditions. Existing asymptotically optimal BC algorithms require solving an optimization problem as part of their sampling rule at each step, which is computationally costly. We propose an Efficient Bandit Clustering algorithm (EBC), which, instead of solving the full optimization problem, takes a single step toward the optimal value at each time step, making it computationally efficient while remaining asymptotically optimal. We also propose a heuristic variant of EBC, called EBC-H, which further simplifies the sampling rule, with arm selection based on quantities computed as part of the stopping rule. We highlight the computational efficiency of EBC and EBC-H by comparing their per-sample run time with that of existing algorithms. The asymptotic optimality of EBC is supported through simulations on the synthetic datasets. Through simulations on both synthetic and real-world datasets, we show the performance gain of EBC and EBC-H over existing approaches.

Efficient Clustering in Stochastic Bandits

TL;DR

This work studies Bandit Clustering (BC) under fixed-confidence for arms with vector-parametric distributions, seeking to recover clustering with minimal samples. It introduces Efficient Bandit Clustering (EBC), which uses a one-step gradient toward the optimal sampling proportions instead of full optimization, and a heuristic EBC-H for further simplification. EBC is shown to be -PC and asymptotically optimal, with strong computational advantages over prior methods; EBC-H matches the asymptotic lower bound slope and often improves practical performance. Empirical results on synthetic and real-world datasets demonstrate that EBC and EBC-H outperform existing approaches in both sample complexity and runtime. Overall, the methods extend BC to broader distribution families and offer scalable, provably efficient clustering in sequential bandit settings.

Abstract

We study the Bandit Clustering (BC) problem under the fixed confidence setting, where the objective is to group a collection of data sequences (arms) into clusters through sequential sampling from adaptively selected arms at each time step while ensuring a fixed error probability at the stopping time. We consider a setting where arms in a cluster may have different distributions. Unlike existing results in this setting, which assume Gaussian-distributed arms, we study a broader class of vector-parametric distributions that satisfy mild regularity conditions. Existing asymptotically optimal BC algorithms require solving an optimization problem as part of their sampling rule at each step, which is computationally costly. We propose an Efficient Bandit Clustering algorithm (EBC), which, instead of solving the full optimization problem, takes a single step toward the optimal value at each time step, making it computationally efficient while remaining asymptotically optimal. We also propose a heuristic variant of EBC, called EBC-H, which further simplifies the sampling rule, with arm selection based on quantities computed as part of the stopping rule. We highlight the computational efficiency of EBC and EBC-H by comparing their per-sample run time with that of existing algorithms. The asymptotic optimality of EBC is supported through simulations on the synthetic datasets. Through simulations on both synthetic and real-world datasets, we show the performance gain of EBC and EBC-H over existing approaches.
Paper Structure (13 sections, 15 theorems, 84 equations, 5 figures, 2 tables, 1 algorithm)

This paper contains 13 sections, 15 theorems, 84 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Lower bound
  4. Efficient Bandit Clustering Algorithm (EBC) and its performance analysis
  5. Efficient Bandit Clustering - Heuristic
  6. Simulation Results
  7. Conclusion
  8. Proof of Theorem 1
  9. Proof of Lemma 1
  10. Proof of Lemma 2
  11. Proof of Lemma 3
  12. Proof of Theorem 2
  13. Proof of Theorem 3

Key Result

Theorem 1

Consider $\delta \in (0, 1)$. For a problem instance $\boldsymbol{\theta} \in \Theta^M$, any $\delta$-PC algorithm $\pi$ (Definition definition: deltapc) satisfies $\mathbb{E}_{\boldsymbol{\theta}}^\pi\left[ \tau_\delta(\pi) \right] \geq \text{KL}(\delta, 1-\delta)T^{*}(\boldsymbol{\theta})$, where Furthermore, $\liminf_{\delta\rightarrow 0}\frac{\mathbb{E}_{\boldsymbol{\theta}}^{\pi}[\tau_\delta

Figures (5)

  • Figure 1: Consider a BC problem with $d=2$, $K=3$, $M=6$, and the parameter vectors $\boldsymbol{\theta} = -1-11233.5-1-212-3-3$. SLINK initially assumes each point as a cluster, as shown in Fig. \ref{['a']}. Then, the two closest clusters are subsequently merged until the number of clusters reaches $K = 3$, as shown in Figs. \ref{['b']}, \ref{['c']}, and \ref{['d']}.
  • Figure 2: Synthetic Dataset 1
  • Figure 3: Synthetic Dataset 2
  • Figure 4: Real-world Dataset 1.
  • Figure 5: Real-world Dataset 2.

Theorems & Definitions (36)

  • Definition 1
  • Theorem 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Remark 1
  • Theorem 2
  • Theorem 3
  • Remark 2
  • proof
  • ...and 26 more