Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploration via Feature Perturbation in Contextual Bandits

Seouh-won Yi, Min-hwan Oh

TL;DR

The paper addresses exploration in contextual bandits by introducing feature perturbation (FP), a simple method that perturbs inputs rather than rewards or model parameters. The proposed GLM-FP algorithm achieves a near-optimal worst-case regret of $\tilde{O}(d\sqrt{\kappa_*T}+d^2/\kappa)$ in generalized linear bandits and extends naturally to non-parametric models and neural networks (DeepFP). The authors provide a rigorous regret analysis, including high-probability parameter confidence sets and concentration/anti-concentration lemmas, and demonstrate strong empirical performance on both generalized linear and neural bandits. FP bridges the gap between deterministic optimism and randomized exploration, offering a scalable, architecture-agnostic approach with practical impact for online decision-making in high-dimensional settings.

Abstract

We propose feature perturbation, a simple yet effective exploration strategy for contextual bandits that injects randomness directly into feature inputs, instead of randomizing unknown parameters or adding noise to rewards. Remarkably, this algorithm achieves $\tilde{\mathcal{O}}(d\sqrt{T})$ worst-case regret bound for generalized linear contextual bandits, while avoiding the $\tilde{\mathcal{O}}(d^{3/2}\sqrt{T})$ regret typical of existing randomized bandit algorithms. Because our algorithm eschews parameter sampling, it is both computationally efficient and naturally extends to non-parametric or neural network models. We verify these advantages through empirical evaluations, demonstrating that feature perturbation not only surpasses existing methods but also unifies strong practical performance with the near-optimal regret guarantees.

Exploration via Feature Perturbation in Contextual Bandits

TL;DR

The paper addresses exploration in contextual bandits by introducing feature perturbation (FP), a simple method that perturbs inputs rather than rewards or model parameters. The proposed GLM-FP algorithm achieves a near-optimal worst-case regret of in generalized linear bandits and extends naturally to non-parametric models and neural networks (DeepFP). The authors provide a rigorous regret analysis, including high-probability parameter confidence sets and concentration/anti-concentration lemmas, and demonstrate strong empirical performance on both generalized linear and neural bandits. FP bridges the gap between deterministic optimism and randomized exploration, offering a scalable, architecture-agnostic approach with practical impact for online decision-making in high-dimensional settings.

Abstract

We propose feature perturbation, a simple yet effective exploration strategy for contextual bandits that injects randomness directly into feature inputs, instead of randomizing unknown parameters or adding noise to rewards. Remarkably, this algorithm achieves worst-case regret bound for generalized linear contextual bandits, while avoiding the regret typical of existing randomized bandit algorithms. Because our algorithm eschews parameter sampling, it is both computationally efficient and naturally extends to non-parametric or neural network models. We verify these advantages through empirical evaluations, demonstrating that feature perturbation not only surpasses existing methods but also unifies strong practical performance with the near-optimal regret guarantees.
Paper Structure (68 sections, 14 theorems, 91 equations, 10 figures, 2 tables, 3 algorithms)

This paper contains 68 sections, 14 theorems, 91 equations, 10 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related works
  3. Preliminaries
  4. Notations.
  5. Generalized linear contextual bandit.
  6. Algorithm: GLM-FP
  7. Extension to general function class
  8. Intuition behind the algorithm
  9. Regret analysis of GLM-FP
  10. Confidence bound for the true parameter
  11. Concentration and anti-concentration
  12. Remark.
  13. Regret bound of GLM-FP
  14. Proof sketch of Theorem 1
  15. Carving off the root d factor compared to TS
  16. ...and 53 more sections

Key Result

Lemma 1

Let $\mathcal{L}_t:=\max_{\theta\in\Theta}\|\nabla L_t(\theta)\|$ denote the Lipschitz constant of the loss function $L_t(\cdot)$.It has been shown by lee2025unified that $\mathcal{L}_t=\mathcal{O}(t)$ for linear, logistic, and Poisson bandit instances. For any $\lambda>0$, define the regularized He

Figures (10)

  • Figure 1: (Left) Model perturbation methods randomize rewards via off-estimated models $\widetilde{f}_t$. (Right) Feature Perturbation (FP) perturbs inputs and evaluates them with a fixed model $\hat{f}_t$.
  • Figure 2: (a) Transformation of the well-explored arm $x_{t1}$ and under-explored arm $x_{t2}$ using $A = D^{1/2}P^\top$, where $V_t^{-1/2}=PDP^\top$. Left: $\zeta_1$ induces a proper bonus. Right: $\zeta_2$ reverses the effect. (b) Average terminal regret $R(T)$ over $100$ runs with $T = 200{,}000$, $K = 50$, and varying $d$.
  • Figure 3: Comparison of cumulative regret across contextual bandit algorithms: linear (top), logistic (middle), and neural (bottom).
  • Figure H.1: Linear Bandit. $|\mathcal{C}|=1$, $d=\{5,10,20\}$, $K=100$, $S=1$.
  • Figure H.2: Linear Bandit. $|\mathcal{C}|=T$, $d=\{10,20,40\}$, $K=100$, $S=2$.
  • ...and 5 more figures

Theorems & Definitions (16)

  • Lemma 1: Adapted from Theorem 3.2. in lee2025unified
  • Definition 1
  • Lemma 2: Concentration
  • Lemma 3: Stochastic optimism
  • Theorem 1
  • Definition B.1: Definition 1. in abeille2017Linear
  • Proposition F.1: Lemma D.1. of lee2025unified
  • Proposition F.2: Taylor's Theorem with Integral Remainder Form
  • Lemma F.1: Elliptical Potential Lemma
  • Lemma F.2: Elliptical Potential Count Lemma
  • ...and 6 more