A Simple, Optimal and Efficient Algorithm for Online Exp-Concave Optimization

TL;DR

LightONS presents a simple yet effective variant of Online Newton Step for online exp-concave optimization. By combining a projection-hysteresis mechanism with an improper-to-proper conversion, it retains $O(d \log T)$ regret while dramatically reducing total runtime to $O(d^2 T + d^{\omega}\sqrt{T \log T})$, and extends to stochastic settings with SXO runtime $\tilde{O}(d^3/\epsilon)$, resolving a COLT'13 open problem. The method preserves the online mirror descent structure, enabling plug-in usage in gradient-norm adaptive regimes, memory-efficient OXO, and parametric stochastic bandits. Empirical results corroborate the theoretical gains with negligible regret deviation from ONS and marked practical speedups. Overall, LightONS advances scalable, geometry-aware online learning for high-dimensional exp-concave problems and broadens the applicability of efficient Hessian-based methods.

Abstract

Online eXp-concave Optimization (OXO) is a fundamental problem in online learning. The standard algorithm, Online Newton Step (ONS), balances statistical optimality and computational practicality, guaranteeing an optimal regret of $O(d \log T)$, where $d$ is the dimension and $T$ is the time horizon. ONS faces a computational bottleneck due to the Mahalanobis projections at each round. This step costs $Ω(d^ω)$ arithmetic operations for bounded domains, even for the unit ball, where $ω\in (2,3]$ is the matrix-multiplication exponent. As a result, the total runtime can reach $\tilde{O}(d^ωT)$, particularly when iterates frequently oscillate near the domain boundary. For Stochastic eXp-concave Optimization (SXO), computational cost is also a challenge. Deploying ONS with online-to-batch conversion for SXO requires $T = \tilde{O}(d/ε)$ rounds to achieve an excess risk of $ε$, and thereby necessitates an $\tilde{O}(d^{ω+1}/ε)$ runtime. A COLT'13 open problem posed by Koren [2013] asks for an SXO algorithm with runtime less than $\tilde{O}(d^{ω+1}/ε)$. This paper proposes a simple variant of ONS, LightONS, which reduces the total runtime to $O(d^2 T + d^ω\sqrt{T \log T})$ while preserving the optimal $O(d \log T)$ regret. LightONS implies an SXO method with runtime $\tilde{O}(d^3/ε)$, thereby answering the open problem. Importantly, LightONS preserves the elegant structure of ONS by leveraging domain-conversion techniques from parameter-free online learning to introduce a hysteresis mechanism that delays expensive Mahalanobis projections until necessary. This design enables LightONS to serve as an efficient plug-in replacement of ONS in broader scenarios, even beyond regret minimization, including gradient-norm adaptive regret, parametric stochastic bandits, and memory-efficient online learning.

Figures (2)

• Figure 1: Experimental results with $\mathcal{X}=\mathcal{B}(1)$, $d=10$, $T=10^4$. The first row shows linear regression with $G=\frac{1}{10}$, $\alpha=5$; The second row shows logistic regression with $G=\frac{1}{10}$, $\alpha=\mathrm{e}^{-1/5}$.
• Figure : ONS journals/ml/HazanAK07

Theorems & Definitions (51)

• Definition 1: exp-concavity
• Proposition 1: Theorem 2 of journals/ml/HazanAK07
• Lemma 1
• Proposition 2: Theorem 9 of COLT23:OQNS
• Lemma 2
• Theorem 1
• Lemma 3: Theorem 2 of ICML20:Ashok
• Lemma 4
• Theorem 2
• Corollary 1