Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Incentivizing Exploration with Linear Contexts and Combinatorial Actions

Mark Sellke

TL;DR

This work gives an analog of this result for linear bandits, where the independence of the prior is replaced by a natural convexity condition, and improves the sample complexity for the pre-Thompson sampling phase of initial data collection in the semibandit model.

Abstract

We advance the study of incentivized bandit exploration, in which arm choices are viewed as recommendations and are required to be Bayesian incentive compatible. Recent work has shown under certain independence assumptions that after collecting enough initial samples, the popular Thompson sampling algorithm becomes incentive compatible. We give an analog of this result for linear bandits, where the independence of the prior is replaced by a natural convexity condition. This opens up the possibility of efficient and regret-optimal incentivized exploration in high-dimensional action spaces. In the semibandit model, we also improve the sample complexity for the pre-Thompson sampling phase of initial data collection.

Incentivizing Exploration with Linear Contexts and Combinatorial Actions

TL;DR

This work gives an analog of this result for linear bandits, where the independence of the prior is replaced by a natural convexity condition, and improves the sample complexity for the pre-Thompson sampling phase of initial data collection in the semibandit model.

Abstract

We advance the study of incentivized bandit exploration, in which arm choices are viewed as recommendations and are required to be Bayesian incentive compatible. Recent work has shown under certain independence assumptions that after collecting enough initial samples, the popular Thompson sampling algorithm becomes incentive compatible. We give an analog of this result for linear bandits, where the independence of the prior is replaced by a natural convexity condition. This opens up the possibility of efficient and regret-optimal incentivized exploration in high-dimensional action spaces. In the semibandit model, we also improve the sample complexity for the pre-Thompson sampling phase of initial data collection.
Paper Structure (19 sections, 19 theorems, 82 equations)

This paper contains 19 sections, 19 theorems, 82 equations.

Table of Contents

  1. Introduction
  2. Problem Formulation and Results
  3. Other Relevant Work
  4. Preliminaries
  5. Bayesian Chernoff Bounds
  6. Linear Bandit
  7. Convex Geometry
  8. Main Result for Linear Bandit
  9. Extension to Generalized Linear Bandit
  10. Two Counterexamples
  11. Initial Exploration for Combinatorial Semibandit
  12. Algorithm \ref{['alg:main']}
  13. The Policy $\pi_j$
  14. Lower Bound
  15. Proofs for Section \ref{['sec:linear']}
  16. ...and 4 more sections

Key Result

Lemma 2.1

Let $\xi\in{\mathbb{R}}^d$ be an unknown parameter and $\gamma$ an observed signal with distribution depending on $\xi$. Suppose there exists an estimator $\theta=\theta(\gamma)\in{\mathbb{R}}^d$ for $\xi$ depending only on this signal, which satisfies for some deterministic $\varepsilon,\delta>0$ t Further, let $\xi\sim \mu$ be generated according to a prior distribution $\mu$, and let $\hat{\xi}

Theorems & Definitions (44)

  • Definition 1
  • Lemma 2.1
  • proof
  • Lemma 2.2: sellke-slivkins
  • Lemma 3.1
  • Definition 2
  • Definition 3
  • Definition 4
  • Lemma 3.2
  • Corollary 3.3
  • ...and 34 more