Prior Diffusiveness and Regret in the Linear-Gaussian Bandit
Yifan Zhu, John C. Duchi, Benjamin Van Roy
TL;DR
This work analyzes Thompson sampling in the linear-Gaussian bandit with Gaussian prior on the coefficient vector, showing that the Bayesian regret decouples into an additive burn-in term and the long-run minimax rate: $\text{Reg}(T)=\widetilde{O}(\sigma d\sqrt{T} + d r \sqrt{\mathrm{Tr}(\Sigma_0)})$. A new generalized elliptical potential lemma provides a flexible tool to bound the cumulative posterior-uncertainty contributions, enabling a tight split between initial exploration (burn-in) and asymptotic performance. The authors also establish a prior-dependent lower bound indicating the burn-in term is unavoidable in general and extend the results to strongly log-concave priors and noise distributions. Overall, the paper clarifies how prior diffusiveness affects regret and demonstrates additive burn-in behavior that improves upon previous multiplicative bounds, with implications for prior design in Bayesian bandits.
Abstract
We prove that Thompson sampling exhibits $\tilde{O}(σd \sqrt{T} + d r \sqrt{\mathrm{Tr}(Σ_0)})$ Bayesian regret in the linear-Gaussian bandit with a $\mathcal{N}(μ_0, Σ_0)$ prior distribution on the coefficients, where $d$ is the dimension, $T$ is the time horizon, $r$ is the maximum $\ell_2$ norm of the actions, and $σ^2$ is the noise variance. In contrast to existing regret bounds, this shows that to within logarithmic factors, the prior-dependent ``burn-in'' term $d r \sqrt{\mathrm{Tr}(Σ_0)}$ decouples additively from the minimax (long run) regret $σd \sqrt{T}$. Previous regret bounds exhibit a multiplicative dependence on these terms. We establish these results via a new ``elliptical potential'' lemma, and also provide a lower bound indicating that the burn-in term is unavoidable.