Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online Learning with Limited Information in the Sliding Window Model

Vladimir Braverman, Sumegha Garg, Chen Wang, David P. Woodruff, Samson Zhou

TL;DR

This work initiates a systematic study of online learning with experts under sliding-window and memory-bounded constraints. It develops a two-query framework that enables near-optimal interval and sliding-window regret, achieving $\sqrt{n|\mathcal{I}|}\,\mathrm{polylog}(nT)$ interval regret with $\mathrm{polylog}(nT)$ space, and thus $\sqrt{nW}\,\mathrm{polylog}(nT)$ sliding-window regret. The paper also extends to the data-stream/bandit setting, delivering a single-query algorithm with $nT^{2/3}\,\mathrm{polylog}(T)$ regret and polylog memory, and a random-order best-expert variant attaining $\mathcal{O}(\sqrt{nT})$ regret under weaker distributional assumptions. A key methodological contribution is decoupling dependencies via fixed second-query randomness, enabling recursive boosting across multiple layers to achieve tight regret while maintaining polylogarithmic memory. These results markedly improve memory efficiency and establish optimal or near-optimal performance for sliding-window learning with limited information, with potential practical impact on time-sensitive applications like traffic monitoring and trading.

Abstract

Motivated by recent work on the experts problem in the streaming model, we consider the experts problem in the sliding window model. The sliding window model is a well-studied model that captures applications such as traffic monitoring, epidemic tracking, and automated trading, where recent information is more valuable than older data. Formally, we have $n$ experts, $T$ days, the ability to query the predictions of $q$ experts on each day, a limited amount of memory, and should achieve the (near-)optimal regret $\sqrt{nW}\text{polylog}(nT)$ regret over any window of the last $W$ days. While it is impossible to achieve such regret with $1$ query, we show that with $2$ queries we can achieve such regret and with only $\text{polylog}(nT)$ bits of memory. Not only are our algorithms optimal for sliding windows, but we also show for every interval $\mathcal{I}$ of days that we achieve $\sqrt{n|\mathcal{I}|}\text{polylog}(nT)$ regret with $2$ queries and only $\text{polylog}(nT)$ bits of memory, providing an exponential improvement on the memory of previous interval regret algorithms. Building upon these techniques, we address the bandit problem in data streams, where $q=1$, achieving $n T^{2/3}\text{polylog}(T)$ regret with $\text{polylog}(nT)$ memory, which is the first sublinear regret in the streaming model in the bandit setting with polylogarithmic memory; this can be further improved to the optimal $\mathcal{O}(\sqrt{nT})$ regret if the best expert's losses are in a random order.

Online Learning with Limited Information in the Sliding Window Model

TL;DR

This work initiates a systematic study of online learning with experts under sliding-window and memory-bounded constraints. It develops a two-query framework that enables near-optimal interval and sliding-window regret, achieving interval regret with space, and thus sliding-window regret. The paper also extends to the data-stream/bandit setting, delivering a single-query algorithm with regret and polylog memory, and a random-order best-expert variant attaining regret under weaker distributional assumptions. A key methodological contribution is decoupling dependencies via fixed second-query randomness, enabling recursive boosting across multiple layers to achieve tight regret while maintaining polylogarithmic memory. These results markedly improve memory efficiency and establish optimal or near-optimal performance for sliding-window learning with limited information, with potential practical impact on time-sensitive applications like traffic monitoring and trading.

Abstract

Motivated by recent work on the experts problem in the streaming model, we consider the experts problem in the sliding window model. The sliding window model is a well-studied model that captures applications such as traffic monitoring, epidemic tracking, and automated trading, where recent information is more valuable than older data. Formally, we have experts, days, the ability to query the predictions of experts on each day, a limited amount of memory, and should achieve the (near-)optimal regret regret over any window of the last days. While it is impossible to achieve such regret with query, we show that with queries we can achieve such regret and with only bits of memory. Not only are our algorithms optimal for sliding windows, but we also show for every interval of days that we achieve regret with queries and only bits of memory, providing an exponential improvement on the memory of previous interval regret algorithms. Building upon these techniques, we address the bandit problem in data streams, where , achieving regret with memory, which is the first sublinear regret in the streaming model in the bandit setting with polylogarithmic memory; this can be further improved to the optimal regret if the best expert's losses are in a random order.
Paper Structure (49 sections, 55 theorems, 137 equations, 1 figure, 22 algorithms)

This paper contains 49 sections, 55 theorems, 137 equations, 1 figure, 22 algorithms.

Table of Contents

  1. Introduction
  2. The sliding window model and interval regret.
  3. Our Contributions
  4. Technical Overview
  5. Shortcomings of Natural Approaches
  6. Histogram frameworks do not work.
  7. Online coresets and importance sampling do not work.
  8. Adaptations of interval regret algorithms.
  9. Adaptation of memory-bounded algorithms.
  10. The Sliding Window Algorithm
  11. Boosting with additional queries.
  12. Removing dependencies for two-query algorithm.
  13. From two-query framework to the sliding-window algorithm.
  14. The Streaming Algorithm in the Single-Query Setting
  15. Boosting with a better balancing.
  16. ...and 34 more sections

Key Result

Theorem 1

There exists an online learning algorithm that given any instance of $n$ experts and $T$ days such that $T\geqslant n$ and two queries per time, achieves $\sqrt{n \left\vert{{\mathcal{I}}\xspace}\right\vert}\cdot \mathop{\mathrm{polylog}}\limits(T)$ interval regret for any interval ${\mathcal{I}}\xs

Figures (1)

  • Figure 1: An illustration of the dependency issue and the ideas to overcome the problem.

Theorems & Definitions (98)

  • Theorem 1
  • Corollary 2
  • Theorem 3
  • Theorem 4
  • Proposition 3.1: Bernstein's inequality, bernstein1924modification
  • Proposition 3.2
  • Proposition 3.3
  • Corollary 5
  • Proposition 3.4
  • Lemma 3.1
  • ...and 88 more