UCB for Large-Scale Pure Exploration: Beyond Sub-Gaussianity
Zaile Li, Weiwei Fan, L. Jeff Hong
TL;DR
This work develops a unified, distribution-free analysis of Upper Confidence Bound (UCB) algorithms for large-scale pure exploration beyond sub-Gaussianity. By introducing a decoupled meta-UCB framework and a boundary-crossing perspective, the authors derive a distribution-free PCS lower bound and prove sample optimality under indifference-zone formulations for both location-scale and bounded-m q-th moment settings, including non-IZ scenarios. The results show that simple UCB variants with per-arm bonuses can achieve optimal sampling budgets even with heavy-tailed distributions, and numerical experiments underscore the role of decoupling and boundary-crossing dynamics in achieving efficiency. The study thus extends the applicability of UCB-based pure exploration to large-scale, non-sub-Gaussian problems with practical implications for robust decision-making under uncertainty.
Abstract
Selecting the best alternative from a finite set represents a broad class of pure exploration problems. Traditional approaches to pure exploration have predominantly relied on Gaussian or sub-Gaussian assumptions on the performance distributions of all alternatives, which limit their applicability to non-sub-Gaussian especially heavy-tailed problems. The need to move beyond sub-Gaussianity may become even more critical in large-scale problems, which tend to be especially sensitive to distributional specifications. In this paper, motivated by the widespread use of upper confidence bound (UCB) algorithms in pure exploration and beyond, we investigate their performance in the large-scale, non-sub-Gaussian settings. We consider the simplest category of UCB algorithms, where the UCB value for each alternative is defined as the sample mean plus an exploration bonus that depends only on its own sample size. We abstract this into a meta-UCB algorithm and propose letting it select the alternative with the largest sample size as the best upon stopping. For this meta-UCB algorithm, we first derive a distribution-free lower bound on the probability of correct selection. Building on this bound, we analyze two general non-sub-Gaussian scenarios: (1) all alternatives follow a common location-scale structure and have bounded variance; and (2) when such a structure does not hold, each alternative has a bounded absolute moment of order $q > 3$. In both settings, we show that the meta-UCB algorithm and therefore a broad class of UCB algorithms can achieve the sample optimality. These results demonstrate the applicability of UCB algorithms for solving large-scale pure exploration problems with non-sub-Gaussian distributions. Numerical experiments support our results and provide additional insights into the comparative behaviors of UCB algorithms within and beyond our meta-UCB framework.