Adaptivity and Universality: Problem-dependent Universal Regret for Online Convex Optimization
Peng Zhao, Yu-Hu Yan, Hang Yu, Zhi-Hua Zhou
TL;DR
This work addresses universal online learning in Online Convex Optimization when curvature is unknown, introducing UniGrad to achieve both universality and problem-dependent adaptivity to gradient variation. It presents two core realizations, UniGrad.Correct and UniGrad.Bregman, each delivering gradient-variation regret bounds that scale with V_T across strongly convex, exp-concave, and convex losses; UniGrad.Correct emphasizes stability cancellation via a cascaded three-layer ensemble, while UniGrad.Bregman leverages negative Bregman divergence to attain optimal convex rates. A shared meta-ensemble (MoM) and a surrogate-optimization extension (UniGrad++) further yield a one-gradient-per-round variant with comparable guarantees, and an anytime extension removes dependence on the horizon T. The results unify universal rates with gradient-variation adaptivity, enabling small-loss and gradient-variance guarantees, SEA-model compatibility, and faster convergence in online games, with practical implications for adversarial-stochastic hybrids and dynamic environments. Overall, UniGrad advances parameter-free, horizon-free, and gradient-variation-aware online learning, offering robust theoretical guarantees and broad applicability across OCO and game-theoretic settings.
Abstract
Universal online learning aims to achieve optimal regret guarantees without requiring prior knowledge of the curvature of online functions. Existing methods have established minimax-optimal regret bounds for universal online learning, where a single algorithm can simultaneously attain $\mathcal{O}(\sqrt{T})$ regret for convex functions, $\mathcal{O}(d \log T)$ for exp-concave functions, and $\mathcal{O}(\log T)$ for strongly convex functions, where $T$ is the number of rounds and $d$ is the dimension of the feasible domain. However, these methods still lack problem-dependent adaptivity. In particular, no universal method provides regret bounds that scale with the gradient variation $V_T$, a key quantity that plays a crucial role in applications such as stochastic optimization and fast-rate convergence in games. In this work, we introduce UniGrad, a novel approach that achieves both universality and adaptivity, with two distinct realizations: UniGrad.Correct and UniGrad.Bregman. Both methods achieve universal regret guarantees that adapt to gradient variation, simultaneously attaining $\mathcal{O}(\log V_T)$ regret for strongly convex functions and $\mathcal{O}(d \log V_T)$ regret for exp-concave functions. For convex functions, the regret bounds differ: UniGrad.Correct achieves an $\mathcal{O}(\sqrt{V_T \log V_T})$ bound while preserving the RVU property that is crucial for fast convergence in online games, whereas UniGrad.Bregman achieves the optimal $\mathcal{O}(\sqrt{V_T})$ regret bound through a novel design. Both methods employ a meta algorithm with $\mathcal{O}(\log T)$ base learners, which naturally requires $\mathcal{O}(\log T)$ gradient queries per round. To enhance computational efficiency, we introduce UniGrad++, which retains the regret while reducing the gradient query to just $1$ per round via surrogate optimization. We further provide various implications.