A/B Testing and Best-arm Identification for Linear Bandits with Robustness to Non-stationarity

TL;DR

The paper addresses fixed-budget best-arm identification in linear bandits under potential non-stationarity of the environment, proposing $\mathsf{P1}$-$\mathsf{RAGE}$ to combine robustness to time-variation with fast identification rates. It introduces a G-optimal-based baseline (G-BAI) for non-stationary settings and shows minimax-like guarantees, then designs $\mathsf{P1}$-$\mathsf{RAGE}$ to achieve near-optimal performance in both stationary and non-stationary regimes. Theoretical results provide stationary bounds with an explicit extra term and non-stationary bounds that match standard non-stationary rates, while experiments demonstrate stable, superior performance of the proposed robust methods across diverse scenarios. The work advances practical adaptive experimentation in non-stationary A/B testing for high-dimensional, covariate-rich linear bandits, with implications for scalable, robust decision making under changing environments.

Abstract

We investigate the fixed-budget best-arm identification (BAI) problem for linear bandits in a potentially non-stationary environment. Given a finite arm set $\mathcal{X}\subset\mathbb{R}^d$, a fixed budget $T$, and an unpredictable sequence of parameters $\left\lbraceθ_t\right\rbrace_{t=1}^{T}$, an algorithm will aim to correctly identify the best arm $x^* := \arg\max_{x\in\mathcal{X}}x^\top\sum_{t=1}^{T}θ_t$ with probability as high as possible. Prior work has addressed the stationary setting where $θ_t = θ_1$ for all $t$ and demonstrated that the error probability decreases as $\exp(-T /ρ^*)$ for a problem-dependent constant $ρ^*$. But in many real-world $A/B/n$ multivariate testing scenarios that motivate our work, the environment is non-stationary and an algorithm expecting a stationary setting can easily fail. For robust identification, it is well-known that if arms are chosen randomly and non-adaptively from a G-optimal design over $\mathcal{X}$ at each time then the error probability decreases as $\exp(-TΔ^2_{(1)}/d)$, where $Δ_{(1)} = \min_{x \neq x^*} (x^* - x)^\top \frac{1}{T}\sum_{t=1}^T θ_t$. As there exist environments where $Δ_{(1)}^2/ d \ll 1/ ρ^*$, we are motivated to propose a novel algorithm $\mathsf{P1}$-$\mathsf{RAGE}$ that aims to obtain the best of both worlds: robustness to non-stationarity and fast rates of identification in benign settings. We characterize the error probability of $\mathsf{P1}$-$\mathsf{RAGE}$ and demonstrate empirically that the algorithm indeed never performs worse than G-optimal design but compares favorably to the best algorithms in the stationary setting.

Figures (6)

• Figure 1: General protocol of fixed-budget best-arm identification (BAI) for linear bandits.
• Figure 2: The vertical axis is on log scale and the shaded area represents the $95\%$ confidence interval.
• Figure 3: Each error probability is estimated through 1000 repeated trials. The bottom two plots give the minimum gap $\Delta_{(1)}$ of each instance as a function of oscillation scale $s$ and oscillation period $L$.
• Figure 4: The error probabilities are estimated through 1000 repeated trials and the error bars represent $95\%$ confidence intervals.
• Figure 5: The error probabilities are estimated through $10^4$ repeated trials and the error bars represent $95\%$ confidence intervals.

Theorems & Definitions (23)

• Remark 1: Comparison to the adversarial setting
• Theorem 1: Error probability of G-BAI
• Theorem 2: Error Probability of P1-RAGE
• Remark 2
• Remark 3
• Theorem 2: Error probability of G-BAI
• proof
• Theorem 3
• proof
• Lemma 1