Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Oracle-efficient Hybrid Learning with Constrained Adversaries

Princewill Okoroafor, Robert Kleinberg, Michael P. Kim

TL;DR

The main result is a new learning algorithm, which runs efficiently given an ERM oracle and obtains regret scaling with the Rademacher complexity of a class derived from the Learner's hypothesis class $H$ and the Adversary's label class $R$.

Abstract

The Hybrid Online Learning Problem, where features are drawn i.i.d. from an unknown distribution but labels are generated adversarially, is a well-motivated setting positioned between statistical and fully-adversarial online learning. Prior work has presented a dichotomy: algorithms that are statistically-optimal, but computationally intractable (Wu et al., 2023), and algorithms that are computationally-efficient (given an ERM oracle), but statistically-suboptimal (Wu et al., 2024). This paper takes a significant step towards achieving statistical optimality and computational efficiency simultaneously in the Hybrid Learning setting. To do so, we consider a structured setting, where the Adversary is constrained to pick labels from an expressive, but fixed, class of functions $R$. Our main result is a new learning algorithm, which runs efficiently given an ERM oracle and obtains regret scaling with the Rademacher complexity of a class derived from the Learner's hypothesis class $H$ and the Adversary's label class $R$. As a key corollary, we give an oracle-efficient algorithm for computing equilibria in stochastic zero-sum games when action sets may be high-dimensional but the payoff function exhibits a type of low-dimensional structure. Technically, we develop a number of tools for the design and analysis of our learning algorithm, including a novel Frank-Wolfe reduction with "truncated entropy regularizer" and a new tail bound for sums of "hybrid" martingale difference sequences.

Oracle-efficient Hybrid Learning with Constrained Adversaries

TL;DR

The main result is a new learning algorithm, which runs efficiently given an ERM oracle and obtains regret scaling with the Rademacher complexity of a class derived from the Learner's hypothesis class and the Adversary's label class .

Abstract

The Hybrid Online Learning Problem, where features are drawn i.i.d. from an unknown distribution but labels are generated adversarially, is a well-motivated setting positioned between statistical and fully-adversarial online learning. Prior work has presented a dichotomy: algorithms that are statistically-optimal, but computationally intractable (Wu et al., 2023), and algorithms that are computationally-efficient (given an ERM oracle), but statistically-suboptimal (Wu et al., 2024). This paper takes a significant step towards achieving statistical optimality and computational efficiency simultaneously in the Hybrid Learning setting. To do so, we consider a structured setting, where the Adversary is constrained to pick labels from an expressive, but fixed, class of functions . Our main result is a new learning algorithm, which runs efficiently given an ERM oracle and obtains regret scaling with the Rademacher complexity of a class derived from the Learner's hypothesis class and the Adversary's label class . As a key corollary, we give an oracle-efficient algorithm for computing equilibria in stochastic zero-sum games when action sets may be high-dimensional but the payoff function exhibits a type of low-dimensional structure. Technically, we develop a number of tools for the design and analysis of our learning algorithm, including a novel Frank-Wolfe reduction with "truncated entropy regularizer" and a new tail bound for sums of "hybrid" martingale difference sequences.
Paper Structure (23 sections, 22 theorems, 93 equations, 2 algorithms)

This paper contains 23 sections, 22 theorems, 93 equations, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Overview of Results
  4. Technical Overview
  5. Technical Preliminaries
  6. Complexity Measures
  7. Linear Optimization Oracle
  8. Comparison to Prior Work
  9. Oracle-Efficient Hybrid Learning
  10. In-Expectation Regret Guarantee using Truncated Entropy Regularization
  11. Overview of \ref{['alg:eg_hybrid']}
  12. Proof of \ref{['thm:main-regret']}
  13. Frank-Wolfe Reduction to Linear Optimization Oracle
  14. Overview of \ref{['alg:hybrid-frank-wolfe']}:
  15. Application to Games
  16. ...and 8 more sections

Key Result

Theorem 1.1

Let $\mathcal{H} \subseteq [0,1]^\mathcal{X}$ be a class of hypothesis functions and let $\mathcal{R} \subseteq [0,1]^\mathcal{X}$ be a class of labeling functions. Let $\ell: [0,1] \times [0,1] \to \mathbb{R}$ be a convex, $L$-Lipschitz loss function in its first argument. There exists an online al As defined in sec:prelim, $\mathsf{rad}_T (\mathcal{F})$ is at most 1 for any $\mathcal{F}$ and $O\

Theorems & Definitions (35)

  • Theorem 1.1
  • Corollary 1.1
  • Proposition 1.1
  • Definition 2.1
  • Theorem 2.1
  • Lemma 2.1: Approximate FTRL for Hybrid Learning
  • Lemma 2.1
  • proof : Proof of \ref{['thm:eg-hybrid']}
  • proof
  • Lemma 3.1: Frank-Wolfe for smooth loss functions
  • ...and 25 more