Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimal Clustering with Bandit Feedback

Junwen Yang, Zixin Zhong, Vincent Y. F. Tan

TL;DR

This work introduces online clustering with bandit feedback, where M arms are partitioned into K unknown clusters and observations are Gaussian around cluster centers. They derive an instance-dependent information-theoretic lower bound and propose Bandit Online Clustering (BOC), combining a weighted K-means Maximin estimator, a D-Tracking sampling rule, and a tractable stopping rule to achieve asymptotic optimality. The algorithm avoids NP-hard clustering subroutines in its subroutines, while ensuring fixed-confidence guarantees and efficient computation. Empirical results on synthetic and real data demonstrate that BOC matches the lower bound asymptotically and significantly outperforms non-adaptive baselines, highlighting its practical impact for online market segmentation and related clustering tasks under noise. The framework also lays groundwork for further extensions to approximate clustering, non-asymptotic analysis, and broader clustering paradigms.

Abstract

This paper considers the problem of online clustering with bandit feedback. A set of arms (or items) can be partitioned into various groups that are unknown. Within each group, the observations associated to each of the arms follow the same distribution with the same mean vector. At each time step, the agent queries or pulls an arm and obtains an independent observation from the distribution it is associated to. Subsequent pulls depend on previous ones as well as the previously obtained samples. The agent's task is to uncover the underlying partition of the arms with the least number of arm pulls and with a probability of error not exceeding a prescribed constant $δ$. The problem proposed finds numerous applications from clustering of variants of viruses to online market segmentation. We present an instance-dependent information-theoretic lower bound on the expected sample complexity for this task, and design a computationally efficient and asymptotically optimal algorithm, namely Bandit Online Clustering (BOC). The algorithm includes a novel stopping rule for adaptive sequential testing that circumvents the need to exactly solve any NP-hard weighted clustering problem as its subroutines. We show through extensive simulations on synthetic and real-world datasets that BOC's performance matches the lower bound asymptotically, and significantly outperforms a non-adaptive baseline algorithm.

Optimal Clustering with Bandit Feedback

TL;DR

This work introduces online clustering with bandit feedback, where M arms are partitioned into K unknown clusters and observations are Gaussian around cluster centers. They derive an instance-dependent information-theoretic lower bound and propose Bandit Online Clustering (BOC), combining a weighted K-means Maximin estimator, a D-Tracking sampling rule, and a tractable stopping rule to achieve asymptotic optimality. The algorithm avoids NP-hard clustering subroutines in its subroutines, while ensuring fixed-confidence guarantees and efficient computation. Empirical results on synthetic and real data demonstrate that BOC matches the lower bound asymptotically and significantly outperforms non-adaptive baselines, highlighting its practical impact for online market segmentation and related clustering tasks under noise. The framework also lays groundwork for further extensions to approximate clustering, non-asymptotic analysis, and broader clustering paradigms.

Abstract

This paper considers the problem of online clustering with bandit feedback. A set of arms (or items) can be partitioned into various groups that are unknown. Within each group, the observations associated to each of the arms follow the same distribution with the same mean vector. At each time step, the agent queries or pulls an arm and obtains an independent observation from the distribution it is associated to. Subsequent pulls depend on previous ones as well as the previously obtained samples. The agent's task is to uncover the underlying partition of the arms with the least number of arm pulls and with a probability of error not exceeding a prescribed constant . The problem proposed finds numerous applications from clustering of variants of viruses to online market segmentation. We present an instance-dependent information-theoretic lower bound on the expected sample complexity for this task, and design a computationally efficient and asymptotically optimal algorithm, namely Bandit Online Clustering (BOC). The algorithm includes a novel stopping rule for adaptive sequential testing that circumvents the need to exactly solve any NP-hard weighted clustering problem as its subroutines. We show through extensive simulations on synthetic and real-world datasets that BOC's performance matches the lower bound asymptotically, and significantly outperforms a non-adaptive baseline algorithm.
Paper Structure (44 sections, 20 theorems, 146 equations, 4 figures, 3 tables, 2 algorithms)

This paper contains 44 sections, 20 theorems, 146 equations, 4 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Two Motivating Examples.
  3. Main Contributions.
  4. Literature Review
  5. Clustering and the $K$-means algorithm.
  6. Bandit Algorithms.
  7. Problem Setup and Preliminaries
  8. A Bandit Feedback Model with Cluster Structure.
  9. The Equivalences of Partitions and Instances.
  10. Online Clustering with Bandit Feedback.
  11. Other Notations.
  12. Lower Bound
  13. Algorithm: Bandit Online Clustering
  14. Weighted K-means with Maximin Initialization
  15. Sampling Rule
  16. ...and 29 more sections

Key Result

Theorem 3

For a fixed confidence level $\delta\in (0,1)$ and instance $(c,\mathcal{U})$, any $\delta$-PAC online clustering algorithm satisfies where Furthermore,

Figures (4)

  • Figure 1: Example involving partitioning $6$ sub-groups of customers into $3$ market segments with bandit feedback. Initially, customers are preliminarily divided into sub-groups, treated as arms, based on some basic characteristics such as age and gender. At each time step, an algorithm chooses a sub-group to query and receives a multidimensional sample (e.g., product ratings) from a single customer within that sub-group. Based on the previously chosen sub-groups and their samples, the algorithm decides which sub-group to query next. Finally, when it is sufficiently confident of producing a partition of the sub-groups into market segments that share similar preferences, the algorithm terminates.
  • Figure 2: Online clustering with bandit feedback with $K=3$ and $M=12$.
  • Figure 3: The empirical averaged sample complexities of the different methods with the two kinds of threshold functions for different confidence levels $\delta$ on the synthetic dataset.
  • Figure 4: An illustration of the proof of Lemma \ref{['theorem_opt1']}. Figure \ref{['figure_lemma1']} illustrates the construction of the sequence of instances when the number of clusters $K = 3$ and the number of arms (or items) $M = 9$. Each pair of items connected by a double arrow represents the cluster indices of one arm in the true partition $c$ and one of the newly constructed partitions $c^{(\bar{k})}$ for $\bar{k}=0,1,2,3$. After the application of the permutation $\sigma$ as defined in \ref{['eqn:permute']} in Step 1, each of the arms can have any cluster index. However, due to the desirable property of the permutation as stated in \ref{['theorem_opt1_firststep']}, in Step 2, we are able to construct a new partition $c^{(1)}$ such that for any arm, its cluster index in $c^{(1)}$ is not larger than that of $c$. Next, we modify the new partition from right to left in Step 3 (see Equations \ref{['eqn:right2left1']} and \ref{['eqn:right2left2']}) and we finally return to a partition that is identical to $c$.

Theorems & Definitions (30)

  • Remark 1
  • Example 1
  • Definition 2
  • Theorem 3
  • Lemma 4
  • Proposition 5
  • Proposition 6
  • Proposition 7
  • Proposition 8
  • Proposition 9
  • ...and 20 more