Improved Dimension Dependence for Bandit Convex Optimization with Gradient Variations
Hang Yu, Yu-Hu Yan, Peng Zhao
TL;DR
The paper studies gradient-variation regret in bandit online learning, focusing on two-point Bandit Convex Optimization (BCO) and the challenge of dimension dependence from non-consecutive gradient variation. By unveiling and decoupling the correlation structure of non-consecutive gradient estimates, the authors derive improved problem-dependent bounds, achieving $\widetilde{\mathcal{O}}(d^{3/2}\sqrt{V_T})$ for convex and $\mathcal{O}(\frac{d}{\lambda}\log V_T)$ for $\lambda$-strongly convex objectives, plus gradient-variance and small-loss guarantees. They extend the framework to one-point BLO, establishing the first gradient-variation bound in that setting, and demonstrate applicability to dynamic/universal regret and bandit games, including a near-optimal duality gap in two-player bandit bilinear games. The results substantially reduce dimension leakage in gradient-variation regret and provide versatile, problem-dependent guarantees across multiple bandit optimization scenarios with practical implications for adaptive strategies. Overall, the work advances the theoretical understanding of gradient-variation in bandit feedback and broadens the tools available for fast, dimension-aware online optimization in adversarial and learning-driven environments.
Abstract
Gradient-variation online learning has drawn increasing attention due to its deep connections to game theory, optimization, etc. It has been studied extensively in the full-information setting, but is underexplored with bandit feedback. In this work, we focus on gradient variation in Bandit Convex Optimization (BCO) with two-point feedback. By proposing a refined analysis on the non-consecutive gradient variation, a fundamental quantity in gradient variation with bandits, we improve the dimension dependence for both convex and strongly convex functions compared with the best known results (Chiang et al., 2013). Our improved analysis for the non-consecutive gradient variation also implies other favorable problem-dependent guarantees, such as gradient-variance and small-loss regrets. Beyond the two-point setup, we demonstrate the versatility of our technique by achieving the first gradient-variation bound for one-point bandit linear optimization over hyper-rectangular domains. Finally, we validate the effectiveness of our results in more challenging tasks such as dynamic/universal regret minimization and bandit games, establishing the first gradient-variation dynamic and universal regret bounds for two-point BCO and fast convergence rates in bandit games.