Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Convex Optimization with Heavy Tails: Old Algorithms, New Regrets, and Applications

Zijian Liu

TL;DR

The paper addresses Online Convex Optimization with heavy-tailed stochastic gradients (p in (1,2]) under a bounded domain and proves that vanilla algorithms OG D, DA, and AdaGrad achieve the optimal in-expectation regret $\mathbb{E}[R_T(x)] \lesssim GD\sqrt{T}+\sigma DT^{1/\mathsf{p}}$ without gradient clipping. It extends these results to online strongly convex, nonsmooth convex, and nonsmooth nonconvex settings, yielding first-optimal convergence rates for nonsmooth convex optimization without clipping and new sample complexities with matching lower bounds for nonsmooth nonconvex optimization via an Online-To-Nonconvex (O2NC) reduction. The framework generalizes to smooth losses and optimistic algorithms, providing adaptive, parameter-free guarantees (notably for AdaGrad) and extending to Hölder smooth and Hölder nonconvex scenarios. These findings bridge theory/practice by explaining why classical OCO methods work under heavy tails and offer practical convergence guarantees with minimal algorithmic changes. The results have implications for stochastic optimization tasks where gradient estimates exhibit heavy-tailed behavior, including robust convergence guarantees without clipping in both convex and certain nonconvex regimes.

Abstract

In Online Convex Optimization (OCO), when the stochastic gradient has a finite variance, many algorithms provably work and guarantee a sublinear regret. However, limited results are known if the gradient estimate has a heavy tail, i.e., the stochastic gradient only admits a finite $\mathsf{p}$-th central moment for some $\mathsf{p}\in\left(1,2\right]$. Motivated by it, this work examines different old algorithms for OCO (e.g., Online Gradient Descent) in the more challenging heavy-tailed setting. Under the standard bounded domain assumption, we establish new regrets for these classical methods without any algorithmic modification. Remarkably, these regret bounds are fully optimal in all parameters (can be achieved even without knowing $\mathsf{p}$), suggesting that OCO with heavy tails can be solved effectively without any extra operation (e.g., gradient clipping). Our new results have several applications. A particularly interesting one is the first provable and optimal convergence result for nonsmooth nonconvex optimization under heavy-tailed noise without gradient clipping. Furthermore, we explore broader settings (e.g., smooth OCO) and extend our ideas to optimistic algorithms to handle different cases simultaneously.

Online Convex Optimization with Heavy Tails: Old Algorithms, New Regrets, and Applications

TL;DR

The paper addresses Online Convex Optimization with heavy-tailed stochastic gradients (p in (1,2]) under a bounded domain and proves that vanilla algorithms OG D, DA, and AdaGrad achieve the optimal in-expectation regret without gradient clipping. It extends these results to online strongly convex, nonsmooth convex, and nonsmooth nonconvex settings, yielding first-optimal convergence rates for nonsmooth convex optimization without clipping and new sample complexities with matching lower bounds for nonsmooth nonconvex optimization via an Online-To-Nonconvex (O2NC) reduction. The framework generalizes to smooth losses and optimistic algorithms, providing adaptive, parameter-free guarantees (notably for AdaGrad) and extending to Hölder smooth and Hölder nonconvex scenarios. These findings bridge theory/practice by explaining why classical OCO methods work under heavy tails and offer practical convergence guarantees with minimal algorithmic changes. The results have implications for stochastic optimization tasks where gradient estimates exhibit heavy-tailed behavior, including robust convergence guarantees without clipping in both convex and certain nonconvex regimes.

Abstract

In Online Convex Optimization (OCO), when the stochastic gradient has a finite variance, many algorithms provably work and guarantee a sublinear regret. However, limited results are known if the gradient estimate has a heavy tail, i.e., the stochastic gradient only admits a finite -th central moment for some . Motivated by it, this work examines different old algorithms for OCO (e.g., Online Gradient Descent) in the more challenging heavy-tailed setting. Under the standard bounded domain assumption, we establish new regrets for these classical methods without any algorithmic modification. Remarkably, these regret bounds are fully optimal in all parameters (can be achieved even without knowing ), suggesting that OCO with heavy tails can be solved effectively without any extra operation (e.g., gradient clipping). Our new results have several applications. A particularly interesting one is the first provable and optimal convergence result for nonsmooth nonconvex optimization under heavy-tailed noise without gradient clipping. Furthermore, we explore broader settings (e.g., smooth OCO) and extend our ideas to optimistic algorithms to handle different cases simultaneously.

Paper Structure

This paper contains 49 sections, 42 theorems, 234 equations, 6 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Discussion on NEURIPS2022_349956de
  4. Preliminary
  5. Old Algorithms under Heavy Tails
  6. New Regret for Online Gradient Descent
  7. New Regret for Dual Averaging
  8. New Regret for $\mathsf{AdaGrad}$
  9. Applications
  10. Nonsmooth Convex Optimization
  11. Nonsmooth Nonconvex Optimization
  12. Online-to-Nonconvex Conversion under Heavy Tails
  13. Convergence Rates
  14. Lower Bounds
  15. Further Extensions
  16. ...and 34 more sections

Key Result

Theorem 1

Under Assumption assu:OCO, taking $\eta_{t}=\frac{D}{G\sqrt{t}}\land\frac{D}{\sigma t^{1/\mathsf{p}}}$ in $\mathsf{OGD}$ (Algorithm alg:OGD), we have

Theorems & Definitions (92)

  • Remark 1
  • Remark 2
  • Remark 3
  • Theorem 1
  • Remark 4
  • Theorem 2
  • Remark 5
  • Theorem 3
  • Remark 6
  • Corollary 1
  • ...and 82 more