Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improved Regret Guarantees for Online Mirror Descent using a Portfolio of Mirror Maps

Swati Gupta, Jai Moondra, Mohit Singh

TL;DR

A meta-algorithm based on multiplicative weights that dynamically selects among a family of uniform block norms is proposed that effectively tunes OMD to the sparsity of the losses, yielding adaptive regret guarantees.

Abstract

OMD and its variants give a flexible framework for OCO where the performance depends crucially on the choice of the mirror map. While the geometries underlying OPGD and OEG, both special cases of OMD, are well understood, it remains a challenging open question on how to construct an optimal mirror map for any given constrained set and a general family of loss functions, e.g., sparse losses. Motivated by parameterizing a near-optimal set of mirror maps, we consider a simpler question: is it even possible to obtain polynomial gains in regret by using mirror maps for geometries that interpolate between $L_1$ and $L_2$, which may not be possible by restricting to only OEG ($L_1$) or OPGD ($L_2$). Our main result answers this question positively. We show that mirror maps based on block norms adapt better to the sparsity of loss functions, compared to previous $L_p$ (for $p \in [1, 2]$) interpolations. In particular, we construct a family of online convex optimization instances in $\mathbb{R}^d$, where block norm-based mirror maps achieve a provable polynomial (in $d$) improvement in regret over OEG and OPGD for sparse loss functions. We then turn to the setting in which the sparsity level of the loss functions is unknown. In this case, the choice of geometry itself becomes an online decision problem. We first show that naively switching between OEG and OPGD can incur linear regret, highlighting the intrinsic difficulty of geometry selection. To overcome this issue, we propose a meta-algorithm based on multiplicative weights that dynamically selects among a family of uniform block norms. We show that this approach effectively tunes OMD to the sparsity of the losses, yielding adaptive regret guarantees. Overall, our results demonstrate that online mirror-map selection can significantly enhance the ability of OMD to exploit sparsity in online convex optimization.

Improved Regret Guarantees for Online Mirror Descent using a Portfolio of Mirror Maps

TL;DR

A meta-algorithm based on multiplicative weights that dynamically selects among a family of uniform block norms is proposed that effectively tunes OMD to the sparsity of the losses, yielding adaptive regret guarantees.

Abstract

OMD and its variants give a flexible framework for OCO where the performance depends crucially on the choice of the mirror map. While the geometries underlying OPGD and OEG, both special cases of OMD, are well understood, it remains a challenging open question on how to construct an optimal mirror map for any given constrained set and a general family of loss functions, e.g., sparse losses. Motivated by parameterizing a near-optimal set of mirror maps, we consider a simpler question: is it even possible to obtain polynomial gains in regret by using mirror maps for geometries that interpolate between and , which may not be possible by restricting to only OEG () or OPGD (). Our main result answers this question positively. We show that mirror maps based on block norms adapt better to the sparsity of loss functions, compared to previous (for ) interpolations. In particular, we construct a family of online convex optimization instances in , where block norm-based mirror maps achieve a provable polynomial (in ) improvement in regret over OEG and OPGD for sparse loss functions. We then turn to the setting in which the sparsity level of the loss functions is unknown. In this case, the choice of geometry itself becomes an online decision problem. We first show that naively switching between OEG and OPGD can incur linear regret, highlighting the intrinsic difficulty of geometry selection. To overcome this issue, we propose a meta-algorithm based on multiplicative weights that dynamically selects among a family of uniform block norms. We show that this approach effectively tunes OMD to the sparsity of the losses, yielding adaptive regret guarantees. Overall, our results demonstrate that online mirror-map selection can significantly enhance the ability of OMD to exploit sparsity in online convex optimization.
Paper Structure (37 sections, 27 theorems, 113 equations, 2 figures, 3 algorithms)

This paper contains 37 sections, 27 theorems, 113 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Key Motivation and Questions:
  3. Technical Details and Formal Statement of Results
  4. Regret Improvement over the Best of OEG and OPGD
  5. Numerical experiment.
  6. Regret lower bound constructions.
  7. Learning the Best Mirror Map Online
  8. Related Work
  9. Outline.
  10. Preliminaries
  11. Online Mirror Descent
  12. Mirror Descent with Block Norms
  13. Mirror Descent Framework.
  14. Block Norms and Improved Regret
  15. Regret for Sparse Gradients
  16. ...and 22 more sections

Key Result

Theorem 1

Given (1) a convex body $\mathcal{K} \subseteq \{x \in \mathbb{R}^d: \|x\|_{[n]} \le 1\}$ that lies within the unit ball of the $n$th block norm for $n \in [d]$, (2) a starting point $x^{(1)} \in \mathcal{K}$, and (3) a sparsity $S \in [d]$, the regret of online mirror descent () with $n$th block no where $D_n := \sqrt{\max_{z \in \mathcal{K}} B_{h_n}(z \| x^{(1)})}$ denotes the diameter under the

Figures (2)

  • Figure 1: A numerical experiment to show the benefit in regret incurred using setups with different block norms. The x-axis shows $13 = 1 + \log_{2} 4096$ block norms, over a 4096-dimensional simplex. The y-axis shows the regret incurred at the end of $T=250$ time steps.
  • Figure 2: Numerical example to show convergence to a suboptimal point (1,0) when the algorithm alternates its choice of mirror map between the Euclidean and entropic maps. The unique optimum for this setting is $x^* = (0,1)$. As the number of iterations increase, the alternating algorithm incurs a linear regret.

Theorems & Definitions (38)

  • Theorem 1
  • Theorem 2: Regret improvements for sparse losses
  • Theorem 3
  • Theorem 4: Learning mirror maps online
  • Corollary 1: Learning block norms online
  • Theorem 5: ben-tal_lectures_2001
  • Corollary 2
  • Theorem 5: Regret improvements for sparse losses
  • Lemma 1
  • Theorem 5
  • ...and 28 more