Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Bound Optimization for Online Convex Optimization

H. Brendan McMahan, Matthew Streeter

TL;DR

This work addresses regret minimization in online convex optimization by introducing FTPRL, an algorithm that adaptively selects proximal regularizers centered at the current iterate. The main technical insight is a regret bound $\text{Regret} \le r_{1:T}(\mathring{x}) + \sum_{t=1}^T \|A_t^{-1} g_t\|^2$ with $A_t=(Q_{1:t})^{1/2}$, enabling competitive guarantees relative to the best post-hoc bound $B_R(\vec{Q_T},\vec{g_T})$. The paper develops several adaptive schemes (coordinate-constant, diagonal, and full-matrix regularization) showing improved, problem-dependent regret bounds on structured feasible sets such as hyperrectangles and ellipsoids, and introduces scale-transformations to extend results to general norm-balls via FTPRL-Scale. Practically, diagonal adaptation yields substantial gains for large-scale, sparse problems (e.g., text classification, CTR prediction) where per-coordinate learning rates align with feature frequency. The results highlight when adaptive regularization helps (notably for hyperrectangles) and point to future work on extending competitive guarantees to arbitrary feasible sets and richer regularization families.

Abstract

We introduce a new online convex optimization algorithm that adaptively chooses its regularization function based on the loss functions observed so far. This is in contrast to previous algorithms that use a fixed regularization function such as L2-squared, and modify it only via a single time-dependent parameter. Our algorithm's regret bounds are worst-case optimal, and for certain realistic classes of loss functions they are much better than existing bounds. These bounds are problem-dependent, which means they can exploit the structure of the actual problem instance. Critically, however, our algorithm does not need to know this structure in advance. Rather, we prove competitive guarantees that show the algorithm provides a bound within a constant factor of the best possible bound (of a certain functional form) in hindsight.

Adaptive Bound Optimization for Online Convex Optimization

TL;DR

This work addresses regret minimization in online convex optimization by introducing FTPRL, an algorithm that adaptively selects proximal regularizers centered at the current iterate. The main technical insight is a regret bound with , enabling competitive guarantees relative to the best post-hoc bound . The paper develops several adaptive schemes (coordinate-constant, diagonal, and full-matrix regularization) showing improved, problem-dependent regret bounds on structured feasible sets such as hyperrectangles and ellipsoids, and introduces scale-transformations to extend results to general norm-balls via FTPRL-Scale. Practically, diagonal adaptation yields substantial gains for large-scale, sparse problems (e.g., text classification, CTR prediction) where per-coordinate learning rates align with feature frequency. The results highlight when adaptive regularization helps (notably for hyperrectangles) and point to future work on extending competitive guarantees to arbitrary feasible sets and richer regularization families.

Abstract

We introduce a new online convex optimization algorithm that adaptively chooses its regularization function based on the loss functions observed so far. This is in contrast to previous algorithms that use a fixed regularization function such as L2-squared, and modify it only via a single time-dependent parameter. Our algorithm's regret bounds are worst-case optimal, and for certain realistic classes of loss functions they are much better than existing bounds. These bounds are problem-dependent, which means they can exploit the structure of the actual problem instance. Critically, however, our algorithm does not need to know this structure in advance. Rather, we prove competitive guarantees that show the algorithm provides a bound within a constant factor of the best possible bound (of a certain functional form) in hindsight.

Paper Structure

This paper contains 15 sections, 15 theorems, 70 equations, 1 figure, 2 algorithms.

Table of Contents

  1. Introduction
  2. Follow the proximally-regularized leader
  3. The practical importance of adaptive regularization
  4. Adaptive algorithms and competitive ratios
  5. Notation and technical background
  6. Analysis of FTPRL
  7. Specific Adaptive Algorithms and Competitive Ratios
  8. Adaptive coordinate-constant regularization
  9. Adaptive diagonal regularization
  10. A post-hoc bound for diagonal regularization on $L_p$ balls
  11. Full matrix regularization on hyperspheres and hyperellipsoids
  12. Related work
  13. Conclusions
  14. A Proof of the FTRL Bound
  15. Proof of Lemma \ref{['lem:sum']}

Key Result

theorem 1

There exists a family of online convex optimization problems, parametrized by the number of rounds $T$, where online subgradient descent with a non-increasing learning rate sequence (and FTPRL with non-decreasing coordinate-constant regularization) incurs regret at least $\Omega(T^\frac{2}{3})$, whe

Figures (1)

  • Figure : FTPRL-Diag

Theorems & Definitions (26)

  • theorem 1
  • theorem 2
  • lemma 1
  • lemma 2
  • proof
  • lemma 3
  • lemma 4
  • proof
  • lemma 5
  • corollary 1
  • ...and 16 more