Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Small Gradient Norm Regret for Online Convex Optimization

Wenzhi Gao, Chang He, Madeleine Udell

TL;DR

It is shown that the new problem-dependent regret measure strictly refines the existing $L^\star$ (small loss) regret, and that it can be arbitrarily sharper when the losses have vanishing curvature around the hindsight decision.

Abstract

This paper introduces a new problem-dependent regret measure for online convex optimization with smooth losses. The notion, which we call the $G^\star$ regret, depends on the cumulative squared gradient norm evaluated at the decision in hindsight $\sum_{t=1}^T \|\nabla \ell(x^\star)\|^2$. We show that the $G^\star$ regret strictly refines the existing $L^\star$ (small loss) regret, and that it can be arbitrarily sharper when the losses have vanishing curvature around the hindsight decision. We establish upper and lower bounds on the $G^\star$ regret and extend our results to dynamic regret and bandit settings. As a byproduct, we refine the existing convergence analysis of stochastic optimization algorithms in the interpolation regime. Some experiments validate our theoretical findings.

Small Gradient Norm Regret for Online Convex Optimization

TL;DR

It is shown that the new problem-dependent regret measure strictly refines the existing (small loss) regret, and that it can be arbitrarily sharper when the losses have vanishing curvature around the hindsight decision.

Abstract

This paper introduces a new problem-dependent regret measure for online convex optimization with smooth losses. The notion, which we call the regret, depends on the cumulative squared gradient norm evaluated at the decision in hindsight . We show that the regret strictly refines the existing (small loss) regret, and that it can be arbitrarily sharper when the losses have vanishing curvature around the hindsight decision. We establish upper and lower bounds on the regret and extend our results to dynamic regret and bandit settings. As a byproduct, we refine the existing convergence analysis of stochastic optimization algorithms in the interpolation regime. Some experiments validate our theoretical findings.
Paper Structure (60 sections, 22 theorems, 92 equations, 1 figure, 1 table, 6 algorithms)

This paper contains 60 sections, 22 theorems, 92 equations, 1 figure, 1 table, 6 algorithms.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related literature
  4. $L^{\star}$ regret.
  5. Gradient variation bound.
  6. Notations
  7. Small gradient norm regret in online convex optimization
  8. $G^{\star}$ regret bound
  9. Connection with existing regret.
  10. Upper and lower bound for the $G^\star$ regret
  11. AdaGrad-Norm.
  12. AdaFTRL for linearized losses.
  13. Lower bound.
  14. Extensions
  15. Dynamic $G^\star$ regret
  16. ...and 45 more sections

Key Result

Proposition 2.1

Under A1 and A2, we always have $G^{\star}_T \leq 2 L (L^{\star}_T)$. Moreover, there exist loss sequences

Figures (1)

  • Figure 1: Comparison between true regret and the theoretical regret upper bounds given by $L_T^\star$ and $G_T^\star$. First row: experiment on $\ell_p$ regression. From left to right: $T \in \{100, 500, 5000, 10000\}$. Second row: experiment on logistic regression. From left to right: $T \in \{100, 500, 5000, 10000\}$.

Theorems & Definitions (38)

  • Remark 1
  • Definition 2.1
  • Remark 2
  • Remark 3
  • Proposition 2.1
  • Example 2.1: Logistic regression with cross-entropy loss
  • Example 2.2: Linear regression with $\ell_p$ loss
  • Example 2.3: Exponential loss
  • Example 2.4: Online regression with vanishing losses
  • Remark 4
  • ...and 28 more