uniINF: Best-of-Both-Worlds Algorithm for Parameter-Free Heavy-Tailed MABs
Yu Chen, Jiatai Huang, Yan Dai, Longbo Huang
TL;DR
The paper tackles heavy-tailed Multi-Armed Bandits in both stochastic and adversarial environments without prior knowledge of heavy-tail parameters, proposing uniINF, a parameter-free BoBW algorithm. It combines Follow-the-Regularized-Leader with a log-barrier regularizer, an auto-balancing learning-rate schedule, and an adaptive skipping-clipping mechanism to handle heavy-tailed losses. Theoretical results show near-optimal regret in adversarial settings, and instance-dependent, near-optimal regret in stochastic settings, with a detailed BoBW regret-decomposition and novel analyses of the Bregman divergence and shifting terms. This work advances robust, parameter-free decision-making under heavy-tailed, time-varying losses, with implications for online learning systems facing unpredictable environments.
Abstract
In this paper, we present a novel algorithm, uniINF, for the Heavy-Tailed Multi-Armed Bandits (HTMAB) problem, demonstrating robustness and adaptability in both stochastic and adversarial environments. Unlike the stochastic MAB setting where loss distributions are stationary with time, our study extends to the adversarial setup, where losses are generated from heavy-tailed distributions that depend on both arms and time. Our novel algorithm `uniINF` enjoys the so-called Best-of-Both-Worlds (BoBW) property, performing optimally in both stochastic and adversarial environments without knowing the exact environment type. Moreover, our algorithm also possesses a Parameter-Free feature, i.e., it operates without the need of knowing the heavy-tail parameters $(σ, α)$ a-priori. To be precise, uniINF ensures nearly-optimal regret in both stochastic and adversarial environments, matching the corresponding lower bounds when $(σ, α)$ is known (up to logarithmic factors). To our knowledge, uniINF is the first parameter-free algorithm to achieve the BoBW property for the heavy-tailed MAB problem. Technically, we develop innovative techniques to achieve BoBW guarantees for Parameter-Free HTMABs, including a refined analysis for the dynamics of log-barrier, an auto-balancing learning rate scheduling scheme, an adaptive skipping-clipping loss tuning technique, and a stopping-time analysis for logarithmic regret.