Stochastic Bandits Robust to Adversarial Attacks
Xuchuang Wang, Jinhang Zuo, Xutong Liu, John C. S. Lui, Mohammad Hajiesmaili
TL;DR
We tackle stochastic MAB under a strong adversary who observes the pulled arm and perturbs the reward, introducing an attack budget $C$ and two knowledge regimes (known vs unknown $C$) with additive and multiplicative regret dependencies. The paper designs robust algorithms—SE-WR and SE-WR-Stop for known $C$, and PE-WR plus MS-SE-WR for unknown $C$—and proves tight upper bounds that scale gracefully with $C$ (e.g., $O\bigl( \sum_{k\neq k^*} \frac{\log T}{\Delta_k} + KC \bigr)$ and $\tilde{O}(\sqrt{KT}\!+\!KC^2)$ in various regimes, up to logarithmic factors). It also establishes matching lower bounds and a fundamental separation between attack and corruption models, showing that attacks can incur larger regret than corruptions under equivalent budgets. The results provide a comprehensive, nearly optimal toolkit for robust bandit learning under adversarial perturbations, with practical guidance on when additive vs multiplicative $C$-dependence is preferable. Together, these findings advance understanding of robustness in online learning under strong adversaries and quantify the cost of attacks in stochastic-bandit settings.
Abstract
This paper investigates stochastic multi-armed bandit algorithms that are robust to adversarial attacks, where an attacker can first observe the learner's action and {then} alter their reward observation. We study two cases of this model, with or without the knowledge of an attack budget $C$, defined as an upper bound of the summation of the difference between the actual and altered rewards. For both cases, we devise two types of algorithms with regret bounds having additive or multiplicative $C$ dependence terms. For the known attack budget case, we prove our algorithms achieve the regret bound of ${O}((K/Δ)\log T + KC)$ and $\tilde{O}(\sqrt{KTC})$ for the additive and multiplicative $C$ terms, respectively, where $K$ is the number of arms, $T$ is the time horizon, $Δ$ is the gap between the expected rewards of the optimal arm and the second-best arm, and $\tilde{O}$ hides the logarithmic factors. For the unknown case, we prove our algorithms achieve the regret bound of $\tilde{O}(\sqrt{KT} + KC^2)$ and $\tilde{O}(KC\sqrt{T})$ for the additive and multiplicative $C$ terms, respectively. In addition to these upper bound results, we provide several lower bounds showing the tightness of our bounds and the optimality of our algorithms. These results delineate an intrinsic separation between the bandits with attacks and corruption models [Lykouris et al., 2018].