Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Hidden Cost of Approximation in Online Mirror Descent

Ofir Schlisselberg, Uri Sherman, Tomer Koren, Yishay Mansour

TL;DR

<3-5 sentence high-level summary> The paper analyzes how approximation errors in Online Mirror Descent affect regret, revealing a fundamental link between the regularizer's geometry (smooth vs barrier) and robustness to inexact updates. It derives a tight bound for smooth regularizers and a sharp dichotomy for barrier regularizers on the simplex: negative entropy requires exponentially small errors to avoid linear regret, while log-barrier and Tsallis maintain optimal regret with polynomial errors; in stochastic settings, negative entropy regains robustness on the full simplex but not on every subset. A balance framework is developed to connect loss balance, effective smoothness, and trajectory behavior, and the results are extended to FTRL-like updates, with practical implications for choosing regularizers and tolerances in online optimization under both adversarial and stochastic losses.

Abstract

Online mirror descent (OMD) is a fundamental algorithmic paradigm that underlies many algorithms in optimization, machine learning and sequential decision-making. The OMD iterates are defined as solutions to optimization subproblems which, oftentimes, can be solved only approximately, leading to an inexact version of the algorithm. Nonetheless, existing OMD analyses typically assume an idealized error free setting, thereby limiting our understanding of performance guarantees that should be expected in practice. In this work we initiate a systematic study into inexact OMD, and uncover an intricate relation between regularizer smoothness and robustness to approximation errors. When the regularizer is uniformly smooth, we establish a tight bound on the excess regret due to errors. Then, for barrier regularizers over the simplex and its subsets, we identify a sharp separation: negative entropy requires exponentially small errors to avoid linear regret, whereas log-barrier and Tsallis regularizers remain robust even when the errors are only polynomial. Finally, we show that when the losses are stochastic and the domain is the simplex, negative entropy regains robustness-but this property does not extend to all subsets, where exponentially small errors are again necessary to avoid suboptimal regret.

The Hidden Cost of Approximation in Online Mirror Descent

TL;DR

<3-5 sentence high-level summary> The paper analyzes how approximation errors in Online Mirror Descent affect regret, revealing a fundamental link between the regularizer's geometry (smooth vs barrier) and robustness to inexact updates. It derives a tight bound for smooth regularizers and a sharp dichotomy for barrier regularizers on the simplex: negative entropy requires exponentially small errors to avoid linear regret, while log-barrier and Tsallis maintain optimal regret with polynomial errors; in stochastic settings, negative entropy regains robustness on the full simplex but not on every subset. A balance framework is developed to connect loss balance, effective smoothness, and trajectory behavior, and the results are extended to FTRL-like updates, with practical implications for choosing regularizers and tolerances in online optimization under both adversarial and stochastic losses.

Abstract

Online mirror descent (OMD) is a fundamental algorithmic paradigm that underlies many algorithms in optimization, machine learning and sequential decision-making. The OMD iterates are defined as solutions to optimization subproblems which, oftentimes, can be solved only approximately, leading to an inexact version of the algorithm. Nonetheless, existing OMD analyses typically assume an idealized error free setting, thereby limiting our understanding of performance guarantees that should be expected in practice. In this work we initiate a systematic study into inexact OMD, and uncover an intricate relation between regularizer smoothness and robustness to approximation errors. When the regularizer is uniformly smooth, we establish a tight bound on the excess regret due to errors. Then, for barrier regularizers over the simplex and its subsets, we identify a sharp separation: negative entropy requires exponentially small errors to avoid linear regret, whereas log-barrier and Tsallis regularizers remain robust even when the errors are only polynomial. Finally, we show that when the losses are stochastic and the domain is the simplex, negative entropy regains robustness-but this property does not extend to all subsets, where exponentially small errors are again necessary to avoid suboptimal regret.

Paper Structure

This paper contains 33 sections, 51 theorems, 176 equations, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Summary of contributions
  3. Related work
  4. Preliminaries
  5. Inexact Online Mirror Descent.
  6. Barrier Regularization.
  7. Additional notation.
  8. Smooth regularizers
  9. Barrier regularizers
  10. Adversarial losses
  11. Improved robustness with stochastic losses
  12. Analysis overview
  13. Balance
  14. Non-entropy barriers: Proof of \ref{['thm:ub_nonentropy']}
  15. Negative entropy
  16. ...and 18 more sections

Key Result

Theorem 2

Let $\mathcal{K} \subseteq \mathbb{R}^d$ be a convex set with diameter $D$, and let $R \colon\mathcal{K}\to \mathbb{R}$ be a $\beta$-smooth regularizer over $\mathcal{K}$. Then, for any loss sequence $\ell_1, \dots, \ell_T \in [-1,1]^d$, the regret of any $\varepsilon$-approximate OMD trajectory wit

Theorems & Definitions (102)

  • Definition 1: coordinate separable barrier regularizers
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Theorem 6
  • Theorem 7
  • Theorem 8
  • Definition 9
  • Lemma 10
  • ...and 92 more