Fast Rates in Stochastic Online Convex Optimization by Exploiting the Curvature of Feasible Sets
Taira Tsuchiya, Shinji Ito
TL;DR
The paper introduces a sphere-enclosed, local curvature condition on the feasible set and couples it with universal online learning to obtain fast regret rates in online convex optimization. By leveraging both the curvature of loss functions and the local curvature of the feasible set, it achieves logarithmic regret in stochastic environments and robust rates in corrupted stochastic and adversarial settings, while extending to $q$-uniformly convex sets to interpolate between $O( ext{log}~T)$ and $O( ext{sqrt}~T)$. It also provides lower bounds that nearly match the upper bounds in key settings, and shows the results extend to polytopes and other locally curved geometries. The framework offers a principled way to exploit curvature locally around the optimal decision, enabling fast rates under milder, local conditions rather than global strong convexity. This has potential practical impact for online learning in constrained convex domains where the local geometry around the optimum is rich but global curvature is limited.
Abstract
In this work, we explore online convex optimization (OCO) and introduce a new condition and analysis that provides fast rates by exploiting the curvature of feasible sets. In online linear optimization, it is known that if the average gradient of loss functions exceeds a certain threshold, the curvature of feasible sets can be exploited by the follow-the-leader (FTL) algorithm to achieve a logarithmic regret. This study reveals that algorithms adaptive to the curvature of loss functions can also leverage the curvature of feasible sets. In particular, we first prove that if an optimal decision is on the boundary of a feasible set and the gradient of an underlying loss function is non-zero, then the algorithm achieves a regret bound of $O(ρ\log T)$ in stochastic environments. Here, $ρ> 0$ is the radius of the smallest sphere that includes the optimal decision and encloses the feasible set. Our approach, unlike existing ones, can work directly with convex loss functions, exploiting the curvature of loss functions simultaneously, and can achieve the logarithmic regret only with a local property of feasible sets. Additionally, the algorithm achieves an $O(\sqrt{T})$ regret even in adversarial environments, in which FTL suffers an $Ω(T)$ regret, and achieves an $O(ρ\log T + \sqrt{C ρ\log T})$ regret in corrupted stochastic environments with corruption level $C$. Furthermore, by extending our analysis, we establish a matching regret upper bound of $O\Big(T^{\frac{q-2}{2(q-1)}} (\log T)^{\frac{q}{2(q-1)}}\Big)$ for $q$-uniformly convex feasible sets, where uniformly convex sets include strongly convex sets and $\ell_p$-balls for $p \in [2,\infty)$. This bound bridges the gap between the $O(\log T)$ bound for strongly convex sets~($q=2$) and the $O(\sqrt{T})$ bound for non-curved sets~($q\to\infty$).