Catoni-Style Change Point Detection for Regret Minimization in Non-Stationary Heavy-Tailed Bandits

TL;DR

This work tackles regret minimization in heavy-tailed piecewise-stationary bandits (HTPS MABs) by introducing Catoni-FCS-detector, a Catoni-style confidence-sequence–based change-point detector, and Robust-CPD-UCB, a meta-algorithm that combines a stationary regret-minimizer with CPD and cyclic exploration. The authors prove a minimax lower bound for HTPS bandits and establish near-optimal regret guarantees for Robust-CPD-UCB, including instance-dependent and instance-independent regimes; they also demonstrate finite-time detection delays under infinite variance. The framework is validated through extensive experiments on synthetic and real-world data, including financial/crypto five datasets, showing robust performance under heavy tails and frequent changes. The results advance practical regret minimization in non-stationary, heavy-tailed environments, enabling reliable learning in finance, communications, and related domains.

Abstract

Regret minimization in stochastic non-stationary bandits gained popularity over the last decade, as it can model a broad class of real-world problems, from advertising to recommendation systems. Existing literature relies on various assumptions about the reward-generating process, such as Bernoulli or subgaussian rewards. However, in settings such as finance and telecommunications, heavy-tailed distributions naturally arise. In this work, we tackle the heavy-tailed piecewise-stationary bandit problem. Heavy-tailed bandits, introduced by Bubeck et al., 2013, operate on the minimal assumption that the finite absolute centered moments of maximum order $1+ε$ are uniformly bounded by a constant $v<+\infty$, for some $ε\in (0,1]$. We focus on the most popular non-stationary bandit setting, i.e., the piecewise-stationary setting, in which the mean of reward-generating distributions may change at unknown time steps. We provide a novel Catoni-style change-point detection strategy tailored for heavy-tailed distributions that relies on recent advancements in the theory of sequential estimation, which is of independent interest. We introduce Robust-CPD-UCB, which combines this change-point detection strategy with optimistic algorithms for bandits, providing its regret upper bound and an impossibility result on the minimum attainable regret for any policy. Finally, we validate our approach through numerical experiments on synthetic and real-world datasets.

Figures (11)

• Figure 1: Rescaled closing prices of four selected cryptocurrencies (blue) with a piecewise-constant approximation (red). Each time step is a day starting in April $2016$. Source: https://www.kaggle.com/datasets/sudalairajkumar/cryptocurrencypricehistory?select=coin_Ethereum.csv.
• Figure 2: Cumulative regrets on HTPS built from cryptocurrency dataset. $20$ trials, mean $\pm$ std.
• Figure 3: Cumulative regrets. $20$ trials, mean $\pm$ std.
• Figure 4: Distribution delay distribution over $20$ trials.
• Figure 5: Cumulative regrets of the considered algorithms. We performed $20$ trials for each instance and reported mean $\pm$ std. The $6$ change-points are indicated by the vertical lines.

Theorems & Definitions (32)

• Definition 2.1: Heavy-Tailed MAB
• Definition 2.2: Piecewise-Stationary Bandit
• Definition 2.3: Heavy-Tailed Piecewise-Stationary Bandits
• Definition 2.4: Expected Cumulative Regret
• Theorem 2.5: Regret Lower Bound for the HTPS Bandit Problem
• Definition 3.1: repeated-FCS-detector, shekhar2023reducing
• Theorem 3.2: Guarantees of repeated-FCS-detector, shekhar2023reducing
• Definition 4.1: Catoni-FCS-detector
• Proposition 4.1: Detection Delay of Catoni-FCS-detector
• Remark 4.2: Comparison with Existing CPDs