Multiclass Online Learnability under Bandit Feedback
Ananth Raman, Vinod Raman, Unique Subedi, Idan Mehalel, Ambuj Tewari
TL;DR
This work characterizes online multiclass learnability under bandit feedback with unbounded label spaces by establishing that the Bandit Littlestone dimension $BL(\mathcal{H})<\infty$ is necessary and sufficient for sublinear regret in both realizable and agnostic settings. It proves a tight, label-space-independent upper bound on regret when $BL(\mathcal{H})<\infty$, via a reduction to a finite-label projection and an EXP4-based strategy, yielding $R_T = O\big(\sqrt{L(\mathcal{H})\,BL(\mathcal{H})\,T\,\log T}\big)$. The paper also shows that Sequential Uniform Convergence is necessary but not sufficient for bandit online learnability, highlighting a separation between SUC and learnability in the bandit band. Finally, it provides lower bounds demonstrating the necessity of finite BLdim and discusses open questions on tightening bounds and exploring randomized dimensions for bandit feedback.
Abstract
We study online multiclass classification under bandit feedback. We extend the results of Daniely and Helbertal [2013] by showing that the finiteness of the Bandit Littlestone dimension is necessary and sufficient for bandit online learnability even when the label space is unbounded. Moreover, we show that, unlike the full-information setting, sequential uniform convergence is necessary but not sufficient for bandit online learnability. Our result complements the recent work by Hanneke, Moran, Raman, Subedi, and Tewari [2023] who show that the Littlestone dimension characterizes online multiclass learnability in the full-information setting even when the label space is unbounded.