Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the necessity of adaptive regularisation:Optimal anytime online learning on $\boldsymbol{\ell_p}$-balls

Emmeran Johnson, David Martínez-Rubio, Ciara Pike-Burke, Patrick Rebeschini

TL;DR

This work analyzes online convex optimization over ℓ_p-balls with p>2, revealing a phase transition in optimal regret between low- and high-dimensional regimes. It shows that FTRL with adaptive regularisation—switching from a uniformly-convex degree-p regulariser in early rounds to a strongly-convex degree-2 regulariser around time t0 ≈ d—achieves anytime-optimal regret without knowledge of the horizon T. A core contribution is a negative result: among separable regularisers, adaptivity is necessary for anytime optimality; fixed separable regularisers cannot achieve the best possible rate in both dimension regimes. The paper also provides lower bounds and bandit-feedback results, demonstrating linear regret in high dimensions and highlighting the fundamental role of dimension and geometry in OCO, with practical implications for designing robust online learners.

Abstract

We study online convex optimization on $\ell_p$-balls in $\mathbb{R}^d$ for $p > 2$. While always sub-linear, the optimal regret exhibits a shift between the high-dimensional setting ($d > T$), when the dimension $d$ is greater than the time horizon $T$ and the low-dimensional setting ($d \leq T$). We show that Follow-the-Regularised-Leader (FTRL) with time-varying regularisation which is adaptive to the dimension regime is anytime optimal for all dimension regimes. Motivated by this, we ask whether it is possible to obtain anytime optimality of FTRL with fixed non-adaptive regularisation. Our main result establishes that for separable regularisers, adaptivity in the regulariser is necessary, and that any fixed regulariser will be sub-optimal in one of the two dimension regimes. Finally, we provide lower bounds which rule out sub-linear regret bounds for the linear bandit problem in sufficiently high-dimension for all $\ell_p$-balls with $p \geq 1$.

On the necessity of adaptive regularisation:Optimal anytime online learning on $\boldsymbol{\ell_p}$-balls

TL;DR

This work analyzes online convex optimization over ℓ_p-balls with p>2, revealing a phase transition in optimal regret between low- and high-dimensional regimes. It shows that FTRL with adaptive regularisation—switching from a uniformly-convex degree-p regulariser in early rounds to a strongly-convex degree-2 regulariser around time t0 ≈ d—achieves anytime-optimal regret without knowledge of the horizon T. A core contribution is a negative result: among separable regularisers, adaptivity is necessary for anytime optimality; fixed separable regularisers cannot achieve the best possible rate in both dimension regimes. The paper also provides lower bounds and bandit-feedback results, demonstrating linear regret in high dimensions and highlighting the fundamental role of dimension and geometry in OCO, with practical implications for designing robust online learners.

Abstract

We study online convex optimization on -balls in for . While always sub-linear, the optimal regret exhibits a shift between the high-dimensional setting (), when the dimension is greater than the time horizon and the low-dimensional setting (). We show that Follow-the-Regularised-Leader (FTRL) with time-varying regularisation which is adaptive to the dimension regime is anytime optimal for all dimension regimes. Motivated by this, we ask whether it is possible to obtain anytime optimality of FTRL with fixed non-adaptive regularisation. Our main result establishes that for separable regularisers, adaptivity in the regulariser is necessary, and that any fixed regulariser will be sub-optimal in one of the two dimension regimes. Finally, we provide lower bounds which rule out sub-linear regret bounds for the linear bandit problem in sufficiently high-dimension for all -balls with .

Paper Structure

This paper contains 59 sections, 37 theorems, 156 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related works
  4. Notation
  5. Preliminaries
  6. FTRL and uniformly-convex regularisation
  7. Low-dimensional regime
  8. High-dimensional regime
  9. Anytime optimality through adaptive regularisation
  10. Necessity of adaptive regularisation
  11. Failure of strong convexity in high-dimensions
  12. Failure of uniform convexity in low-dimension
  13. Failure of fixed separable regularisation
  14. Proof intuitions
  15. Bandit feedback
  16. ...and 44 more sections

Key Result

Theorem 2.3

Let $V \subset \mathbb{R}^d$ be convex and consider proper convex losses $(\ell_t)_{t=1}^T$. For $t \geq 1$, let $\psi_t : \mathbb{R}^d \rightarrow \mathbb{R}$ be a proper, closed and differentiable $\mu_t$-uniformly-convex function on $V$ of degree $r_t \geq 2$ with respect to a norm $\|\cdot\|_{|t Then for any $u \in V$ and $g_t \in \partial \ell_t(x_t)$, the points played by FTRL satisfy the fo

Figures (1)

  • Figure 1: Comparison of FTRL with different regularisation. We fix $T = 40$ (and $L = 1$, $p =10$) and vary the dimension. FTRL - $\phi_2$ refers to FTRL using the regulariser $\phi_2 = \frac{1}{2} \|x\|_2^2$ from \ref{['sec:OPT_low_dim']} with $\eta_{t-1} = \sqrt{\frac{d^{1-2/p}}{2t}}$. FTRL - $\phi_p$ refers to FTRL using the regulariser $\phi_p = \frac{1}{p} \|x\|_p^p$ from \ref{['sec:OPT_high_dim']} with $\eta_{t-1} = \frac{1}{(2p_\star t)^{1/p_\star}}$. The final two correspond to using the procedure from \ref{['sec:anytime_bounds_through_adaptive_regularisation']} with adaptive regularisation. The first with the threshold $t_0 = 3^{-2p/(p-2)} d$ from \ref{['thm:anytime_optimal_regret']}, while the second uses the threshold $t_0 = 2d$.

Theorems & Definitions (55)

  • Remark 2.1
  • Definition 2.2: nesterov2018lectures
  • Theorem 2.3
  • Corollary 2.4
  • Theorem 2.5
  • Proposition 2.6
  • Theorem 2.7
  • Theorem 3.1
  • Proposition 4.1
  • Theorem 4.2
  • ...and 45 more