On Universally Optimal Algorithms for A/B Testing

TL;DR

The paper studies Fixed-Budget Best-Arm Identification in stochastic bandits with Bernoulli rewards, focusing on the A/B testing two-arm case. It proves that no adaptive algorithm strictly beats the uniform sampling strategy on all instances, by introducing consistent and stable algorithms and deriving a matching instance-specific lower bound liminf_{T→∞} T / log(1/p_{μ,T}) ≥ 1 / g(1/2, μ). The authors also analyze the multi-arm setting, providing a tight asymptotic characterization of the Successive Rejects algorithm and showing that, in some instances, uniform sampling outperforms SR. Collectively, the results establish universal optimality of uniform sampling for the two-arm problem and advance understanding of FB-BAI with more arms, highlighting limits of adaptivity and directions for broader reward distributions.

Abstract

We study the problem of best-arm identification with fixed budget in stochastic multi-armed bandits with Bernoulli rewards. For the problem with two arms, also known as the A/B testing problem, we prove that there is no algorithm that (i) performs as well as the algorithm sampling each arm equally (referred to as the {\it uniform sampling} algorithm) in all instances, and that (ii) strictly outperforms uniform sampling on at least one instance. In short, there is no algorithm better than the uniform sampling algorithm. To establish this result, we first introduce the natural class of {\it consistent} and {\it stable} algorithms, and show that any algorithm that performs as well as the uniform sampling algorithm in all instances belongs to this class. The proof then proceeds by deriving a lower bound on the error rate satisfied by any consistent and stable algorithm, and by showing that the uniform sampling algorithm matches this lower bound. Our results provide a solution to the two open problems presented in \citep{qin2022open}. For the general problem with more than two arms, we provide a first set of results. We characterize the asymptotic error rate of the celebrated Successive Rejects (SR) algorithm \citep{audibert2010best} and show that, surprisingly, the uniform sampling algorithm outperforms the SR algorithm in some instances.

Figures (6)

• Figure 1: Left: Visualization of Lemma \ref{['lem:prop g']} with $a=0.55$ and $x=0.51$. The blue region indicates where $\mu_1>\mu_2$ and $g(x,\boldsymbol{\mu})<g(1/2,\boldsymbol{\mu})$. The red curve represents $\lambda(x,\boldsymbol{\mu})=a$. The intersection of the blue region and red curve validates Lemma \ref{['lem:prop g']}. Right: Visualization of Proposition \ref{['prop:intersection']} with $a=0.55$ and $x=0.51$. The green region indicates (i)$\mu_1>\mu_2,\,\mu_1+\mu_2\ge 1$. The red curve represents (ii)$\lambda(x,\boldsymbol{\mu})=a$. The blue region shows (iii)$x^*(\boldsymbol{\mu})< (\frac{1}{2}+x)/2$. The intersection of the three regions validates Proposition \ref{['prop:intersection']}.
• Figure 2: Visualization of the function $g(x, \boldsymbol{\mu})$ properties. The left panel shows the partition of $\Lambda$ into four regions by $\mu_1=\mu_2$ and $\mu_1+\mu_2=1$, with blue indicating $x^*(\boldsymbol{\mu})<1/2$ and red indicating $x^*(\boldsymbol{\mu})>1/2$. Four bandit instances are chosen symmetrically from these regions for further analysis. The right panels show the functions $g(x,\boldsymbol{\mu}^{(A)})=g(x,\boldsymbol{\mu}^{(C)})$ (top) and $g(x,\boldsymbol{\mu}^{(B)})=g(x,\boldsymbol{\mu}^{(D)})$ (bottom), demonstrating the asymmetrical property as stated in Proposition \ref{['prop:asymmetry']}.
• Figure 3: The blue area corresponds to instances $\boldsymbol{\mu}=(\mu_1,\mu_2,\mu_2)$ such that $\mu_1>\mu_2=\mu_3$ and such that the uniform sampling algorithm strictly outperforms the SR algorithm. The red dashed line is the set of instances such that $\mu_1=\mu_2$.
• Figure 4: Visualization of Lemma \ref{['lem:intersection dual']} with $a=0.55$ and $x=0.51$. The green region indicates $(\overline{\textnormal{i}})\,\xi_1>\xi_2,\xi_1\ge -\xi_2$. The red curve represents $(\overline{\textnormal{ii}})\, (1-x)\xi_1+x\xi_2=\alpha$. The blue region shows $(\overline{\textnormal{iii}})\, \bar{d}(\xi_1,(1-\widetilde{x})\xi_1+\widetilde{x}\xi_2) >\bar{d}(\xi_2,(1-\widetilde{x})\xi_1+\widetilde{x}\xi_2)$, where $\widetilde{x}=(\frac{1}{2}+x)/2$.
• Figure 5: Error probability comparison across algorithms for varying sample budgets ($T=6000$ to $T=40000$). Derived from $40000$ trials for each setting and algorithm.

Theorems & Definitions (53)

• Definition 2.1
• Theorem 2.2
• Definition 3.1
• Definition 3.2
• Theorem 3.3
• Lemma 3.4
• Lemma 3.5
• proof : Proof of Lemma \ref{['lem:prop g']}
• Proposition 3.6
• Proposition 3.7