Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sequential Probability Assignment with Contexts: Minimax Regret, Contextual Shtarkov Sums, and Contextual Normalized Maximum Likelihood

Ziyi Liu, Idan Attias, Daniel M. Roy

TL;DR

This work extends sequential probability assignment to contextual, multiary settings by introducing contextual Shtarkov sums and proving that the minimax regret is the worst-case log contextual Shtarkov sum. It then derives contextual Normalized Maximum Likelihood (cNML) as the minimax optimal predictor, applicable to arbitrary finite label sets and time-varying contexts. The authors also show that contextual Shtarkov sums yield sharp regret bounds via sequential entropy, and they demonstrate both the strengths and limitations of existing sequential covering approaches. Together, these results provide a unified, minimax-optimal framework for online probability forecasting with log loss in rich contextual environments and set the stage for future computational and theoretical refinements.

Abstract

We study the fundamental problem of sequential probability assignment, also known as online learning with logarithmic loss, with respect to an arbitrary, possibly nonparametric hypothesis class. Our goal is to obtain a complexity measure for the hypothesis class that characterizes the minimax regret and to determine a general, minimax optimal algorithm. Notably, the sequential $\ell_{\infty}$ entropy, extensively studied in the literature (Rakhlin and Sridharan, 2015, Bilodeau et al., 2020, Wu et al., 2023), was shown to not characterize minimax risk in general. Inspired by the seminal work of Shtarkov (1987) and Rakhlin, Sridharan, and Tewari (2010), we introduce a novel complexity measure, the \emph{contextual Shtarkov sum}, corresponding to the Shtarkov sum after projection onto a multiary context tree, and show that the worst case log contextual Shtarkov sum equals the minimax regret. Using the contextual Shtarkov sum, we derive the minimax optimal strategy, dubbed \emph{contextual Normalized Maximum Likelihood} (cNML). Our results hold for sequential experts, beyond binary labels, which are settings rarely considered in prior work. To illustrate the utility of this characterization, we provide a short proof of a new regret upper bound in terms of sequential $\ell_{\infty}$ entropy, unifying and sharpening state-of-the-art bounds by Bilodeau et al. (2020) and Wu et al. (2023).

Sequential Probability Assignment with Contexts: Minimax Regret, Contextual Shtarkov Sums, and Contextual Normalized Maximum Likelihood

TL;DR

This work extends sequential probability assignment to contextual, multiary settings by introducing contextual Shtarkov sums and proving that the minimax regret is the worst-case log contextual Shtarkov sum. It then derives contextual Normalized Maximum Likelihood (cNML) as the minimax optimal predictor, applicable to arbitrary finite label sets and time-varying contexts. The authors also show that contextual Shtarkov sums yield sharp regret bounds via sequential entropy, and they demonstrate both the strengths and limitations of existing sequential covering approaches. Together, these results provide a unified, minimax-optimal framework for online probability forecasting with log loss in rich contextual environments and set the stage for future computational and theoretical refinements.

Abstract

We study the fundamental problem of sequential probability assignment, also known as online learning with logarithmic loss, with respect to an arbitrary, possibly nonparametric hypothesis class. Our goal is to obtain a complexity measure for the hypothesis class that characterizes the minimax regret and to determine a general, minimax optimal algorithm. Notably, the sequential entropy, extensively studied in the literature (Rakhlin and Sridharan, 2015, Bilodeau et al., 2020, Wu et al., 2023), was shown to not characterize minimax risk in general. Inspired by the seminal work of Shtarkov (1987) and Rakhlin, Sridharan, and Tewari (2010), we introduce a novel complexity measure, the \emph{contextual Shtarkov sum}, corresponding to the Shtarkov sum after projection onto a multiary context tree, and show that the worst case log contextual Shtarkov sum equals the minimax regret. Using the contextual Shtarkov sum, we derive the minimax optimal strategy, dubbed \emph{contextual Normalized Maximum Likelihood} (cNML). Our results hold for sequential experts, beyond binary labels, which are settings rarely considered in prior work. To illustrate the utility of this characterization, we provide a short proof of a new regret upper bound in terms of sequential entropy, unifying and sharpening state-of-the-art bounds by Bilodeau et al. (2020) and Wu et al. (2023).
Paper Structure (34 sections, 22 theorems, 127 equations, 1 algorithm)

This paper contains 34 sections, 22 theorems, 127 equations, 1 algorithm.

Table of Contents

  1. Introduction
  2. Main contributions.
  3. Related work.
  4. Preliminaries
  5. Notation.
  6. Sequential probability assignment and minimax regret.
  7. Multiary trees.
  8. Time-varying context sets.
  9. Prior work: the Shtarkov sum in context-free and fixed designs
  10. Minimax regret via contextual Shtarkov sum
  11. Proof sketch.
  12. Applications: an improved regret upper bound in terms of sequential entropy
  13. The inadequacy of sequential text covering number
  14. Contextual NML, the minimax optimal algorithm
  15. Perspectives on contextual Shtarkov sums
  16. ...and 19 more sections

Key Result

Proposition 2.3

In the context-free setting, for any hypothesis class $\mathcal{F}$ and horizon $T$, the Shtarkov sum $S_{T}(\mathcal{F})$ is defined as Moreover, the minimax regret is given by $\mathcal{R}_T(\mathcal{F}) = \log S_{T}(\mathcal{F})$, and the unique minimax optimal strategy is the normalized maximum likelihood (NML) distribution given by

Theorems & Definitions (49)

  • Remark 2.1: Sequential vs non-sequential experts
  • Definition 2.2: Likelihood
  • Proposition 2.3: Sht87
  • Definition 2.4: Conditional Shtarkov sum
  • Proposition 2.5: Minimax regret, fixed design jacquet2021precise
  • Definition 3.1: Contextual Shtarkov sum
  • theorem 3.2: Main result: minimax regret
  • Proposition 3.3: Finite classes
  • Definition 3.4: Sequential $\ell_{\infty}$ cover and entropy
  • Definition 3.5: Global sequential $\ell_{\infty}$ cover and entropy
  • ...and 39 more