Improved Regret Analysis in Gaussian Process Bandits: Optimality for Noiseless Reward, RKHS norm, and Non-Stationary Variance
Shogo Iwazaki, Shion Takeno
TL;DR
This work advances GP-bandit theory by deriving a tighter uniform bound on the maximum posterior variance that improves how regret scales with noise. Leveraging this bound, it achieves near-optimal regret in the noiseless setting, establishes regret that scales optimally with the RKHS norm bound $B$ for both squared exponential and Matérn kernels, and extends the analysis to a kernelized non-stationary variance setting with sublinear variance growth. The proposed variance-aware variants of PE and MVR (and VA-GP-UCB) yield regret rates that match known lower bounds up to polylogarithmic factors, underscoring near-optimality and broad applicability to kernelized optimization under challenging noise regimes.
Abstract
We study the Gaussian process (GP) bandit problem, whose goal is to minimize regret under an unknown reward function lying in some reproducing kernel Hilbert space (RKHS). The maximum posterior variance analysis is vital in analyzing near-optimal GP bandit algorithms such as maximum variance reduction (MVR) and phased elimination (PE). Therefore, we first show the new upper bound of the maximum posterior variance, which improves the dependence of the noise variance parameters of the GP. By leveraging this result, we refine the MVR and PE to obtain (i) a nearly optimal regret upper bound in the noiseless setting and (ii) regret upper bounds that are optimal with respect to the RKHS norm of the reward function. Furthermore, as another application of our proposed bound, we analyze the GP bandit under the time-varying noise variance setting, which is the kernelized extension of the linear bandit with heteroscedastic noise. For this problem, we show that MVR and PE-based algorithms achieve noise variance-dependent regret upper bounds, which matches our regret lower bound.