A Complete Characterization of Learnability for Stochastic Noisy Bandits
Steve Hanneke, Kun Wang
TL;DR
We address learnability in stochastic noisy bandits with an unknown reward function drawn from a known function class. The authors introduce the generalized maximin volume $\gamma_{\mathcal{F},\alpha}$ and prove a necessary-and-sufficient condition: the function class is learnable under arbitrary noise if and only if $\gamma_{\mathcal{F},\alpha}>0$ for all $\alpha\in(0,1)$, and they characterize the full range of optimal query complexities, including adaptive vs. non-adaptive regimes. The work extends the theory to unbounded and Gaussian noise via median-of-means techniques and proposes a new variant of the Decision Estimation Coefficient based on online regression to capture learnability in this setting. Together, these results resolve the open question of learnability for noisy bandits in full generality and provide actionable guidance on the required query complexity and adaptability for achieving near-optimal performance across diverse noise models.
Abstract
We study the stochastic noisy bandit problem with an unknown reward function $f^*$ in a known function class $\mathcal{F}$. Formally, a model $M$ maps arms $π$ to a probability distribution $M(π)$ of reward. A model class $\mathcal{M}$ is a collection of models. For each model $M$, define its mean reward function $f^M(π)=\mathbb{E}_{r \sim M(π)}[r]$. In the bandit learning problem, we proceed in rounds, pulling one arm $π$ each round and observing a reward sampled from $M(π)$. With knowledge of $\mathcal{M}$, supposing that the true model $M\in \mathcal{M}$, the objective is to identify an arm $\hatπ$ of near-maximal mean reward $f^M(\hatπ)$ with high probability in a bounded number of rounds. If this is possible, then the model class is said to be learnable. Importantly, a result of \cite{hanneke2023bandit} shows there exist model classes for which learnability is undecidable. However, the model class they consider features deterministic rewards, and they raise the question of whether learnability is decidable for classes containing sufficiently noisy models. For the first time, we answer this question in the positive by giving a complete characterization of learnability for model classes with arbitrary noise. In addition to that, we also describe the full spectrum of possible optimal query complexities. Further, we prove adaptivity is sometimes necessary to achieve the optimal query complexity. Last, we revisit an important complexity measure for interactive decision making, the Decision-Estimation-Coefficient \citep{foster2021statistical,foster2023tight}, and propose a new variant of the DEC which also characterizes learnability in this setting.