Efficient Simple Regret Algorithms for Stochastic Contextual Bandits

TL;DR

The paper studies stochastic contextual bandits under a simple-regret objective for both linear and logistic models. It develops both deterministic and randomized online algorithms—MULin, MULog, SimpleLinTS, and THaTS—achieving leading simple-regret bounds that are independent of the logistic curvature constant $\kappa$ in the leading term. Specifically, deterministic logistic methods reach $\tilde{O}(d/\sqrt{T})$ with $\kappa$-free leading terms, while randomized variants achieve $\tilde{O}(d^{3/2}/\sqrt{T})$; all methods remain tractable for finite action sets. The theoretical guarantees are complemented by numerical experiments showing empirical validity and computational benefits of the randomized approaches. Overall, the work provides a fully online, κ-free framework for simple-regret optimality in stochastic contextual bandits with both linear and logistic link functions, with practical implications for online decision making under uncertainty.

Abstract

We study stochastic contextual logistic bandits under the simple regret objective. While simple regret guarantees have been established for the linear case, no such results were previously known for the logistic setting. Building on ideas from contextual linear bandits and self-concordant analysis, we propose the first algorithm that achieves simple regret $\tilde{\mathcal{O}}(d/\sqrt{T})$. Notably, the leading term of our regret bound is free of the constant $κ= \mathcal O(\exp(S))$, where $S$ is a bound on the magnitude of the unknown parameter vector. The algorithm is shown to be fully tractable when the action set is finite. We also introduce a new variant of Thompson Sampling tailored to the simple-regret setting. This yields the first simple regret guarantee for randomized algorithms in stochastic contextual linear bandits, with regret $\tilde{\mathcal{O}}(d^{3/2}/\sqrt{T})$. Extending this method to the logistic case, we obtain a similarly structured Thompson Sampling algorithm that achieves the same regret bound -- $\tilde{\mathcal{O}}(d^{3/2}/\sqrt{T})$ -- again with no dependence on $κ$ in the leading term. The randomized algorithms, as expected, are cheaper to run than their deterministic counterparts. Finally, we conducted a series of experiments to empirically validate these theoretical guarantees.

Figures (9)

• Figure 1: Linear case. The noise is $0$-mean Gaussian with std. dev. $\|\theta_*\|$. Left: Average simple regret vs. number of rounds $T$ for $d=8,K=128$. Right: Number of rounds needed to make the simple regret fall below $0.1$ vs. $K$.
• Figure 2: Left: Logistic case where $M=2$. Average simple regret v.s. number of rounds for $d=10, T=1500, M=2$. Right: Number of rounds needed to make the simple regret fall below $10^{-4}$ v.s. $M$ for $d=50$.
• Figure 3: The estimation of "unimportant" components of $\theta_*$ for $M=2$. Left: $\theta_*[0]$, Right: $\theta_*[1]$.
• Figure 4: The estimation of "unimportant" components of $\theta_*$ for $M=2$. Left: $\theta_*[2]$, Right: $\theta_*[3]$.
• Figure 5: The estimation of "unimportant" components of $\theta_*$ for $M=2$. Left: $\theta_*[4]$, Right: $\theta_*[5]$.

Theorems & Definitions (61)

• Lemma 1
• Theorem 1: MULin Simple Regret Bound
• Lemma 2
• Lemma 3: Decreasing Uncertainty Lemma -- Logistic Bandits
• Theorem 2: MULog Simple Regret Bound
• Lemma 4
• Theorem 3
• Lemma 5
• Lemma 6
• Theorem 4