Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse Optimistic Information Directed Sampling

Ludovic Schwartz, Hamish Flynn, Gergely Neu

TL;DR

Sparse Optimistic Information Directed Sampling (SOIDS) delivers a frequentist, regime-adaptive approach to sparse linear bandits by replacing the Bayesian posterior with a time-adaptive optimistic posterior within the Information-Directed Sampling framework. By carefully balancing surrogate information gain and surrogate regret through time-varying learning rates, SOIDS achieves near-optimal worst-case regret in both data-rich and data-poor regimes, with explicit bounds that interpolate between $\tilde{O}(\sqrt{sdT})$ and $\tilde{O}((sT)^{2/3})$. The analysis hinges on a Follow the Regularized Leader formulation, a subset-selection prior, and a forerunner-based information-ratio argument, yielding instance-dependent guarantees that scale with surrogate information ratios rather than horizon length. Empirically, SOIDS demonstrates strong performance across regimes and offers a practical IDS-inspired alternative with potential advantages over DEC-based methods.

Abstract

Many high-dimensional online decision-making problems can be modeled as stochastic sparse linear bandits. Most existing algorithms are designed to achieve optimal worst-case regret in either the data-rich regime, where polynomial dependence on the ambient dimension is unavoidable, or the data-poor regime, where dimension-independence is possible at the cost of worse dependence on the number of rounds. In contrast, the sparse Information Directed Sampling (IDS) algorithm satisfies a Bayesian regret bound that has the optimal rate in both regimes simultaneously. In this work, we explore the use of Sparse Optimistic Information Directed Sampling (SOIDS) to achieve the same adaptivity in the worst-case setting, without Bayesian assumptions. Through a novel analysis that enables the use of a time-dependent learning rate, we show that SOIDS can optimally balance information and regret. Our results extend the theoretical guarantees of IDS, providing the first algorithm that simultaneously achieves optimal worst-case regret in both the data-rich and data-poor regimes. We empirically demonstrate the good performance of SOIDS.

Sparse Optimistic Information Directed Sampling

TL;DR

Sparse Optimistic Information Directed Sampling (SOIDS) delivers a frequentist, regime-adaptive approach to sparse linear bandits by replacing the Bayesian posterior with a time-adaptive optimistic posterior within the Information-Directed Sampling framework. By carefully balancing surrogate information gain and surrogate regret through time-varying learning rates, SOIDS achieves near-optimal worst-case regret in both data-rich and data-poor regimes, with explicit bounds that interpolate between and . The analysis hinges on a Follow the Regularized Leader formulation, a subset-selection prior, and a forerunner-based information-ratio argument, yielding instance-dependent guarantees that scale with surrogate information ratios rather than horizon length. Empirically, SOIDS demonstrates strong performance across regimes and offers a practical IDS-inspired alternative with potential advantages over DEC-based methods.

Abstract

Many high-dimensional online decision-making problems can be modeled as stochastic sparse linear bandits. Most existing algorithms are designed to achieve optimal worst-case regret in either the data-rich regime, where polynomial dependence on the ambient dimension is unavoidable, or the data-poor regime, where dimension-independence is possible at the cost of worse dependence on the number of rounds. In contrast, the sparse Information Directed Sampling (IDS) algorithm satisfies a Bayesian regret bound that has the optimal rate in both regimes simultaneously. In this work, we explore the use of Sparse Optimistic Information Directed Sampling (SOIDS) to achieve the same adaptivity in the worst-case setting, without Bayesian assumptions. Through a novel analysis that enables the use of a time-dependent learning rate, we show that SOIDS can optimally balance information and regret. Our results extend the theoretical guarantees of IDS, providing the first algorithm that simultaneously achieves optimal worst-case regret in both the data-rich and data-poor regimes. We empirically demonstrate the good performance of SOIDS.

Paper Structure

This paper contains 50 sections, 32 theorems, 156 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Sparse linear bandits.
  4. The data-rich and data-poor regimes.
  5. Adapting to both regimes.
  6. The sparse optimal action condition.
  7. Notation.
  8. Sparse Optimistic Information Directed Sampling
  9. Main results
  10. General bound for the optimistic posterior
  11. Adapting to both regimes
  12. Instance-dependent guarantees
  13. Analysis
  14. Proof of Theorem \ref{['thm:Optimistic_posterior_RB']}
  15. Proof of Theorem \ref{['thm:regret_main']}
  16. ...and 35 more sections

Key Result

Lemma 1

For all $\gamma \geq 2$,

Figures (1)

  • Figure 1: Cumulative regret for $d = 20$ (left) $40$ (middle) and $100$ (right). We plot the mean $\pm$ standard deviation over 10 repetitions.

Theorems & Definitions (49)

  • Definition 1
  • Lemma 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • ...and 39 more