Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Asymptotically-Optimal Gaussian Bandits with Side Observations

Alexia Atsidakou, Orestis Papadigenopoulos, Constantine Caramanis, Sujay Sanghavi, Sanjay Shakkottai

TL;DR

This work advances online learning with rich feedback by addressing Gaussian bandits where a known side-information matrix $\Sigma$ governs cross-arm information. It introduces a weighted maximum-likelihood estimator to fuse heterogeneous noise across samples, and proves martingale-based concentration bounds under adaptive sampling. A key contribution is an LP-based asymptotic regret lower bound that characterizes the minimal exploration cost required to identify the optimal arm when information can be gleaned from other arms. Building on this, the authors propose a general algorithm that interleaves greedy exploitation with two exploration modes (uniform and LP-dictated) and prove it is asymptotically optimal, matching the LP lower bound up to an $\epsilon$-approximation that vanishes as time grows. The results unify and extend previous graph- and full-information models, offering near-optimal regret guarantees for a broad class of feedback structures with practical implications for learning under side observations.

Abstract

We study the problem of Gaussian bandits with general side information, as first introduced by Wu, Szepesvari, and Gyorgy. In this setting, the play of an arm reveals information about other arms, according to an arbitrary a priori known side information matrix: each element of this matrix encodes the fidelity of the information that the ``row'' arm reveals about the ``column'' arm. In the case of Gaussian noise, this model subsumes standard bandits, full-feedback, and graph-structured feedback as special cases. In this work, we first construct an LP-based asymptotic instance-dependent lower bound on the regret. The LP optimizes the cost (regret) required to reliably estimate the suboptimality gap of each arm. This LP lower bound motivates our main contribution: the first known asymptotically optimal algorithm for this general setting.

Asymptotically-Optimal Gaussian Bandits with Side Observations

TL;DR

This work advances online learning with rich feedback by addressing Gaussian bandits where a known side-information matrix governs cross-arm information. It introduces a weighted maximum-likelihood estimator to fuse heterogeneous noise across samples, and proves martingale-based concentration bounds under adaptive sampling. A key contribution is an LP-based asymptotic regret lower bound that characterizes the minimal exploration cost required to identify the optimal arm when information can be gleaned from other arms. Building on this, the authors propose a general algorithm that interleaves greedy exploitation with two exploration modes (uniform and LP-dictated) and prove it is asymptotically optimal, matching the LP lower bound up to an -approximation that vanishes as time grows. The results unify and extend previous graph- and full-information models, offering near-optimal regret guarantees for a broad class of feedback structures with practical implications for learning under side observations.

Abstract

We study the problem of Gaussian bandits with general side information, as first introduced by Wu, Szepesvari, and Gyorgy. In this setting, the play of an arm reveals information about other arms, according to an arbitrary a priori known side information matrix: each element of this matrix encodes the fidelity of the information that the ``row'' arm reveals about the ``column'' arm. In the case of Gaussian noise, this model subsumes standard bandits, full-feedback, and graph-structured feedback as special cases. In this work, we first construct an LP-based asymptotic instance-dependent lower bound on the regret. The LP optimizes the cost (regret) required to reliably estimate the suboptimality gap of each arm. This LP lower bound motivates our main contribution: the first known asymptotically optimal algorithm for this general setting.
Paper Structure (30 sections, 17 theorems, 85 equations)

This paper contains 30 sections, 17 theorems, 85 equations.

Table of Contents

  1. Introduction
  2. Our Contributions
  3. Related Work
  4. Problem Definition
  5. Model.
  6. Maximum-Likelihood Estimator and Concentration Bounds
  7. Remark.
  8. Asymptotic Regret: Lower Bound
  9. Algorithm and Regret Analysis
  10. Description of the Algorithm and Main Results
  11. Greedy exploitation.
  12. Uniform and LP-dictated exploration.
  13. Outline of the Regret Analysis
  14. Rounds satisfying $\mathop{\mathrm{\mathcal{U}_t}}\nolimits,\mathop{\mathrm{\mathcal{A}_t}}\nolimits$.
  15. Rounds satisfying $\mathop{\mathrm{\mathcal{U}_t}}\nolimits,\mathop{\mathrm{\mathcal{A}_t}}\nolimits^c, \mathop{\mathrm{\mathcal{B}_t}}\nolimits$.
  16. ...and 15 more sections

Key Result

Lemma 3.1

Let $\{Z_{t'}\}_{t' \in \mathbb{N}}$ be a sequence of random variables. We denote by $\mathcal{F}_{t'}$ the $\sigma$-algebra generated by $\{Z_{\tau}\}_{\tau \leq t'}$ and by $\mathcal{F} = (\mathcal{F}_{t'})_{{t'} \in \mathbb{N}}$ the corresponding filtration. Each random variable $Z_{t'}$ is drawn

Theorems & Definitions (29)

  • Lemma 3.1
  • proof : Proof sketch.
  • Lemma 3.2
  • Definition 4.1
  • Theorem 4.1
  • Proposition 4.1
  • Definition 5.1
  • Theorem 5.1
  • Corollary 5.1
  • Lemma 5.1
  • ...and 19 more