Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Oracle-Efficient Smoothed Online Learning for Piecewise Continuous Decision Making

Adam Block, Alexander Rakhlin, Max Simchowitz

TL;DR

A new notion of complexity is introduced, the generalized bracketing numbers, which marries constraints on the adversary to the size of the space, and it is shown that an instantiation of Follow-the-Perturbed-Leader can attain low regret with the number of calls to the optimization oracle scaling optimally with respect to average regret.

Abstract

Smoothed online learning has emerged as a popular framework to mitigate the substantial loss in statistical and computational complexity that arises when one moves from classical to adversarial learning. Unfortunately, for some spaces, it has been shown that efficient algorithms suffer an exponentially worse regret than that which is minimax optimal, even when the learner has access to an optimization oracle over the space. To mitigate that exponential dependence, this work introduces a new notion of complexity, the generalized bracketing numbers, which marries constraints on the adversary to the size of the space, and shows that an instantiation of Follow-the-Perturbed-Leader can attain low regret with the number of calls to the optimization oracle scaling optimally with respect to average regret. We then instantiate our bounds in several problems of interest, including online prediction and planning of piecewise continuous functions, which has many applications in fields as diverse as econometrics and robotics.

Oracle-Efficient Smoothed Online Learning for Piecewise Continuous Decision Making

TL;DR

A new notion of complexity is introduced, the generalized bracketing numbers, which marries constraints on the adversary to the size of the space, and it is shown that an instantiation of Follow-the-Perturbed-Leader can attain low regret with the number of calls to the optimization oracle scaling optimally with respect to average regret.

Abstract

Smoothed online learning has emerged as a popular framework to mitigate the substantial loss in statistical and computational complexity that arises when one moves from classical to adversarial learning. Unfortunately, for some spaces, it has been shown that efficient algorithms suffer an exponentially worse regret than that which is minimax optimal, even when the learner has access to an optimization oracle over the space. To mitigate that exponential dependence, this work introduces a new notion of complexity, the generalized bracketing numbers, which marries constraints on the adversary to the size of the space, and shows that an instantiation of Follow-the-Perturbed-Leader can attain low regret with the number of calls to the optimization oracle scaling optimally with respect to average regret. We then instantiate our bounds in several problems of interest, including online prediction and planning of piecewise continuous functions, which has many applications in fields as diverse as econometrics and robotics.
Paper Structure (30 sections, 34 theorems, 144 equations, 2 algorithms)

This paper contains 30 sections, 34 theorems, 144 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Formal Setting and Notation
  3. Notation
  4. Follow the Perturbed Leader and Generalized Brackets
  5. Exponential Perturbations and Piecewise Continuous Functions
  6. Piecewise-Continuous Prediction
  7. Piecewise Continuous Prediction with Generalized Affine Boundaries
  8. Piecewise Continuous Prediction with Polynomial Boundaries
  9. Smoothed Multi-Step Planning
  10. Related Work
  11. Smoothed Online Learning
  12. Follow the Perturbed Leader and Oracle-Efficient Online Learning
  13. Prediction and Planning in Piecewise Affine Systems
  14. Statistical and Online Learning for Control and Dynamical Prediction.
  15. Bracketing Entropy
  16. ...and 15 more sections

Key Result

Proposition 3.1

Let $\mathcal{M}$ and $\rho$ be as in Definition def:genbrackets and suppose that $z_1, \dots, z_n$ are generated such that the law $p_i$ of $z_i$ conditioned on $\sigma$-algebra $\mathscr{F}_i$ generated by the $z_j$ up to time $i$ satisfies $p_i \in \mathcal{M}$ for all $1 \leq i \leq n$. Suppose

Theorems & Definitions (67)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3: From Section 3.5.2 in gine2021mathematical
  • Definition 3.1
  • Proposition 3.1
  • Definition 3.2
  • Proposition 3.2
  • Corollary 3.1
  • Theorem 1
  • Theorem 2
  • ...and 57 more