Rate-optimal Design for Anytime Best Arm Identification

TL;DR

This work tackles anytime best arm identification under a limited sampling budget by introducing Almost Tracking, a horizon-free, non-elimination sampling rule with closed-form allocations. The authors develop a trackability- and optimization-aware framework, proving a constant-factor approximation to the minimax one-shot bound and rate-optimality for the standard $H_1$ risk. They derive a practical closed-form allocation $w^*_i(\bm{Q})$ and establish a batched tracking analysis that preserves samples while ensuring exponential decay of error probability, even without knowing the horizon $T$. Extensive experiments on synthetic minimax benchmarks and real datasets (Open Bandit, MovieLens) show that Almost Tracking and Simple Tracking outperform existing anytime and fixed-budget baselines, highlighting the approach's robustness and practical impact for adaptive experimental design.

Abstract

We consider the best arm identification problem, where the goal is to identify the arm with the highest mean reward from a set of $K$ arms under a limited sampling budget. This problem models many practical scenarios such as A/B testing. We consider a class of algorithms for this problem, which is provably minimax optimal up to a constant factor. This idea is a generalization of existing works in fixed-budget best arm identification, which are limited to a particular choice of risk measures. Based on the framework, we propose Almost Tracking, a closed-form algorithm that has a provable guarantee on the popular risk measure $H_1$. Unlike existing algorithms, Almost Tracking does not require the total budget in advance nor does it need to discard a significant part of samples, which gives a practical advantage. Through experiments on synthetic and real-world datasets, we show that our algorithm outperforms existing anytime algorithms as well as fixed-budget algorithms.

Figures (10)

• Figure 1: Hypothetical $\bm{Q}'$ such that $H_1'(i, \bm{Q}):=H_1(\bm{Q}')$ with $i=2$. $Q_i'=Q_i$ for $i \ne 2$.
• Figure 2: Comparison of PoE across algorithms on real-world datasets (smaller better). For algorithms that do not discard samples, $J(t)$ is the empirical best arm at time $t$. For discarding algorithms (SH, DSH, DSR), $J(t)$ is the empirical best arm at the end of the most recent batch.
• Figure 3: Illustration of the added constraints by Claims 2--4. We use a four-armed instance where the best arm in $\bm{P}$ is arm $j=2$. Claim 2 (left) states that $P_2 > Q_2$ and $P_i \le Q_i$ for $i \ne 2$. Claims 3 (middle) states that bold blue arrow of arm $2$ is longer than the standard-size blue arrow of arm $1$. Claim 4 (right) states that blue arrow of arm $2$ is at most twice as the blue arrow of arm $1$.
• Figure : Instance 1: $\bm{P} = \{1 - (i-1)\times0.05\}$
• Figure : Instance 3: $\bm{P} = \{1 - \sqrt{i-1}/10\}$. This instance is in favor of SR that we discusssed in Section \ref{['sec_srwin']}.

Theorems & Definitions (59)

• Definition 1: Rate
• Definition 2
• Definition 3
• Theorem 1: Theorem 1 in komiyama2022minimax
• Theorem 2
• Theorem 3
• Definition 4
• Theorem 4
• proof : Proof of Theorem \ref{['thm_rateopt_gen']}
• Remark 1