Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastically Constrained Best Arm Identification with Thompson Sampling

Le Yang, Siyang Gao, Cheng Li, Yi Wang

TL;DR

The paper tackles best feasible arm identification under stochastic, multidimensional rewards in a fixed-budget setting. It introduces BFAI-TS, a Thompson Sampling-based top-two algorithm that uses a balancing parameter $β$ to allocate samples between the best-feasible arm and the remaining arms, achieving exponential-rate posterior convergence. Theoretical results establish asymptotically optimal sampling allocations and the best possible convergence rate $Γ_{β^*}$, while extensive simulations and a dose-finding case demonstrate substantial empirical gains over relevant benchmarks. This work advances constrained ranking and selection by providing both rigorous guarantees and practical, scalable procedures for identifying the best feasible option.

Abstract

We consider the problem of the best arm identification in the presence of stochastic constraints, where there is a finite number of arms associated with multiple performance measures. The goal is to identify the arm that optimizes the objective measure subject to constraints on the remaining measures. We will explore the popular idea of Thompson sampling (TS) as a means to solve it. To the best of our knowledge, it is the first attempt to extend TS to this problem. We will design a TS-based sampling algorithm, establish its asymptotic optimality in the rate of posterior convergence, and demonstrate its superior performance using numerical examples.

Stochastically Constrained Best Arm Identification with Thompson Sampling

TL;DR

The paper tackles best feasible arm identification under stochastic, multidimensional rewards in a fixed-budget setting. It introduces BFAI-TS, a Thompson Sampling-based top-two algorithm that uses a balancing parameter to allocate samples between the best-feasible arm and the remaining arms, achieving exponential-rate posterior convergence. Theoretical results establish asymptotically optimal sampling allocations and the best possible convergence rate , while extensive simulations and a dose-finding case demonstrate substantial empirical gains over relevant benchmarks. This work advances constrained ranking and selection by providing both rigorous guarantees and practical, scalable procedures for identifying the best feasible option.

Abstract

We consider the problem of the best arm identification in the presence of stochastic constraints, where there is a finite number of arms associated with multiple performance measures. The goal is to identify the arm that optimizes the objective measure subject to constraints on the remaining measures. We will explore the popular idea of Thompson sampling (TS) as a means to solve it. To the best of our knowledge, it is the first attempt to extend TS to this problem. We will design a TS-based sampling algorithm, establish its asymptotic optimality in the rate of posterior convergence, and demonstrate its superior performance using numerical examples.
Paper Structure (14 sections, 6 theorems, 77 equations, 2 figures, 1 table, 1 algorithm)

This paper contains 14 sections, 6 theorems, 77 equations, 2 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. BFAI-TS Algorithm
  4. Theoretical Results
  5. Numerical Results
  6. Conclusion
  7. Preliminaries
  8. Notations
  9. Important Lemmas
  10. Proof of Theorem 1
  11. Proof of Proposition 1
  12. Proof of Theorem 1
  13. Proof of Theorem 2
  14. Proof of Lemma \ref{['newadd1']}

Key Result

Theorem 4.1

For the BFAI-TS Algorithm, $\mathbb{E}[N_{\beta}^{\epsilon}]<\infty$ for any $\epsilon>0$. The sample allocations of the algorithm is asymptotically optimal in the sense that

Figures (2)

  • Figure 1: PFS and their sampling rates on selected arms (Experiment 1)
  • Figure 4: PFS and their sampling rates on selected arms (Experiment 4)

Theorems & Definitions (6)

  • Theorem 4.1
  • Theorem 4.2
  • Lemma A.1
  • Lemma A.2
  • Proposition B.1
  • Lemma C.1