Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimax Optimal Simple Regret in Two-Armed Best-Arm Identification

Masahiro Kato

TL;DR

This work addresses two-arm fixed-budget best-arm identification with unknown variances. It proposes a Neyman allocation, coupled with adaptive variance estimation and an AIPW estimator for final mean estimation, and proves that this approach attains a tight minimax optimality for the simple regret $Regret_P(\pi)$, with both lower and matching upper bounds: $\inf_{\pi}\liminf_{T\to\infty} \sqrt{T}\sup_{P\in\mathcal{P}_{\bm{\sigma}^2}} Regret_P(\pi) \ge \frac{1}{\sqrt{e}}(\sigma(1)+\sigma(2))$ and $\limsup_{T\to\infty} \sup_{P\in\mathcal{P}_{\bm{\sigma}^2}} \sqrt{T} Regret_P(\pi^{NA}) \le \frac{1}{\sqrt{e}}(\sigma(1)+\sigma(2))$. Notably, the results hold without locality assumptions and extend beyond Gaussian settings; in Bernoulli cases, Neyman coincides with uniform allocation. These findings advance the theory of minimax optimality in adaptive experimental design with unknown variances and non-Gaussian outcomes.

Abstract

This study investigates an asymptotically minimax optimal algorithm in the two-armed fixed-budget best-arm identification (BAI) problem. Given two treatment arms, the objective is to identify the arm with the highest expected outcome through an adaptive experiment. We focus on the Neyman allocation, where treatment arms are allocated following the ratio of their outcome standard deviations. Our primary contribution is to prove the minimax optimality of the Neyman allocation for the simple regret, defined as the difference between the expected outcomes of the true best arm and the estimated best arm. Specifically, we first derive a minimax lower bound for the expected simple regret, which characterizes the worst-case performance achievable under the location-shift distributions, including Gaussian distributions. We then show that the simple regret of the Neyman allocation asymptotically matches this lower bound, including the constant term, not just the rate in terms of the sample size, under the worst-case distribution. Notably, our optimality result holds without imposing locality restrictions on the distribution, such as the local asymptotic normality. Furthermore, we demonstrate that the Neyman allocation reduces to the uniform allocation, i.e., the standard randomized controlled trial, under Bernoulli distributions.

Minimax Optimal Simple Regret in Two-Armed Best-Arm Identification

TL;DR

This work addresses two-arm fixed-budget best-arm identification with unknown variances. It proposes a Neyman allocation, coupled with adaptive variance estimation and an AIPW estimator for final mean estimation, and proves that this approach attains a tight minimax optimality for the simple regret , with both lower and matching upper bounds: and . Notably, the results hold without locality assumptions and extend beyond Gaussian settings; in Bernoulli cases, Neyman coincides with uniform allocation. These findings advance the theory of minimax optimality in adaptive experimental design with unknown variances and non-Gaussian outcomes.

Abstract

This study investigates an asymptotically minimax optimal algorithm in the two-armed fixed-budget best-arm identification (BAI) problem. Given two treatment arms, the objective is to identify the arm with the highest expected outcome through an adaptive experiment. We focus on the Neyman allocation, where treatment arms are allocated following the ratio of their outcome standard deviations. Our primary contribution is to prove the minimax optimality of the Neyman allocation for the simple regret, defined as the difference between the expected outcomes of the true best arm and the estimated best arm. Specifically, we first derive a minimax lower bound for the expected simple regret, which characterizes the worst-case performance achievable under the location-shift distributions, including Gaussian distributions. We then show that the simple regret of the Neyman allocation asymptotically matches this lower bound, including the constant term, not just the rate in terms of the sample size, under the worst-case distribution. Notably, our optimality result holds without imposing locality restrictions on the distribution, such as the local asymptotic normality. Furthermore, we demonstrate that the Neyman allocation reduces to the uniform allocation, i.e., the standard randomized controlled trial, under Bernoulli distributions.

Paper Structure

This paper contains 16 sections, 9 theorems, 46 equations.

Table of Contents

  1. Introduction
  2. Problem setting
  3. Notation.
  4. Content of the Paper
  5. Related work
  6. The Neyman Allocation
  7. The Neyman allocation in the allocation phase
  8. AIPW estimator in the recommendation phase
  9. Statistical models and lower bounds
  10. Location-shift models
  11. Minimax lower bound
  12. Proof of the minimax lower bound
  13. Upper bound and minimax optimality
  14. Proof of Theorem \ref{['thm:upper_simpleregret']}.
  15. Extension to Bernoulli Distributions
  16. ...and 1 more sections

Key Result

Proposition 1.1

Assume that $w^*$ is known. Allocate treatment arm $1$ for the first $T_1 = T w^*(1)$ samples and treatment arm $2$ for the next $T_2 = T w^*(2)$ samples, where $T_1 + T_2 = T$. Using these samples, compute Recommend $\widehat{a}_T^{\dagger} \coloneqq \arg\max_{a \in \{1, 2\}} \widehat{\mu}_T^{\dagger}(a)$ as the best arm. Then, for any Gaussian distribution $P \in \{\mathcal{N}(\mu(1), \sigma^2(

Theorems & Definitions (13)

  • Proposition 1.1: Theorem 12 in Kaufman2016complexity (informal)
  • Definition 3.1: Location-shift models
  • Definition 3.2: Consistent algorithm
  • Theorem 3.3: Lower bounds
  • Proposition 3.4: Transportation lemma. From Lemma 1 in Kaufman2016complexity
  • Proposition 3.5: Proposition 15.3.2. in Duchi2023 and Theorem 4.4.4 in calin2014geometric
  • proof : Proof of Theorem \ref{['thm:minimax_lowerbound']}
  • Theorem 4.1
  • Corollary 4.2: Asymptotic minimax optimality
  • Lemma 4.3
  • ...and 3 more