Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Laplacian Kernelized Bandit

Shuang Wu, Arash A. Amini

TL;DR

This work tackles multi-user contextual bandits where users are connected by a known graph and rewards exhibit non-linear behavior with graph-aware homophily. By regularizing the collection of user functions with a graph smoothness term plus ridge, the authors prove an equivalence to learning a single lifted function in a unified multi-user RKHS, deriving the kernel K((x,u),(x',u')) = [{\boldsymbol L}_\rho^{-1}]_{uu'} K_x(x,x'). They develop LK-GP-UCB and LK-GP-TS algorithms that operate on GP posteriors of this kernel, and provide regret bounds that scale with an effective dimension that captures the interplay between the graph and the base kernel. Their analysis reveals favorable scaling under graph homophily, and experiments show substantial improvements over non-graph-aware baselines in non-linear settings while remaining competitive when rewards are linear. Overall, the paper offers a principled bridge between Laplacian regularization and kernelized bandits for structured exploration with theoretical guarantees and practical performance.

Abstract

We study multi-user contextual bandits where users are related by a graph and their reward functions exhibit both non-linear behavior and graph homophily. We introduce a principled joint penalty for the collection of user reward functions $\{f_u\}$, combining a graph smoothness term based on RKHS distances with an individual roughness penalty. Our central contribution is proving that this penalty is equivalent to the squared norm within a single, unified \emph{multi-user RKHS}. We explicitly derive its reproducing kernel, which elegantly fuses the graph Laplacian with the base arm kernel. This unification allows us to reframe the problem as learning a single ''lifted'' function, enabling the design of principled algorithms, \texttt{LK-GP-UCB} and \texttt{LK-GP-TS}, that leverage Gaussian Process posteriors over this new kernel for exploration. We provide high-probability regret bounds that scale with an \emph{effective dimension} of the multi-user kernel, replacing dependencies on user count or ambient dimension. Empirically, our methods outperform strong linear and non-graph-aware baselines in non-linear settings and remain competitive even when the true rewards are linear. Our work delivers a unified, theoretically grounded, and practical framework that bridges Laplacian regularization with kernelized bandits for structured exploration.

Laplacian Kernelized Bandit

TL;DR

This work tackles multi-user contextual bandits where users are connected by a known graph and rewards exhibit non-linear behavior with graph-aware homophily. By regularizing the collection of user functions with a graph smoothness term plus ridge, the authors prove an equivalence to learning a single lifted function in a unified multi-user RKHS, deriving the kernel K((x,u),(x',u')) = [{\boldsymbol L}_\rho^{-1}]_{uu'} K_x(x,x'). They develop LK-GP-UCB and LK-GP-TS algorithms that operate on GP posteriors of this kernel, and provide regret bounds that scale with an effective dimension that captures the interplay between the graph and the base kernel. Their analysis reveals favorable scaling under graph homophily, and experiments show substantial improvements over non-graph-aware baselines in non-linear settings while remaining competitive when rewards are linear. Overall, the paper offers a principled bridge between Laplacian regularization and kernelized bandits for structured exploration with theoretical guarantees and practical performance.

Abstract

We study multi-user contextual bandits where users are related by a graph and their reward functions exhibit both non-linear behavior and graph homophily. We introduce a principled joint penalty for the collection of user reward functions , combining a graph smoothness term based on RKHS distances with an individual roughness penalty. Our central contribution is proving that this penalty is equivalent to the squared norm within a single, unified \emph{multi-user RKHS}. We explicitly derive its reproducing kernel, which elegantly fuses the graph Laplacian with the base arm kernel. This unification allows us to reframe the problem as learning a single ''lifted'' function, enabling the design of principled algorithms, \texttt{LK-GP-UCB} and \texttt{LK-GP-TS}, that leverage Gaussian Process posteriors over this new kernel for exploration. We provide high-probability regret bounds that scale with an \emph{effective dimension} of the multi-user kernel, replacing dependencies on user count or ambient dimension. Empirically, our methods outperform strong linear and non-graph-aware baselines in non-linear settings and remain competitive even when the true rewards are linear. Our work delivers a unified, theoretically grounded, and practical framework that bridges Laplacian regularization with kernelized bandits for structured exploration.
Paper Structure (50 sections, 11 theorems, 128 equations, 6 figures, 1 table, 2 algorithms)

This paper contains 50 sections, 11 theorems, 128 equations, 6 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Gang of Bandits with Non-Linear Rewards
  4. A Principled Regularity Model for Graph Homophily
  5. From Joint Penalty to a Unified Multi-user Kernel
  6. Laplacian Kernelized Bandit Algorithms
  7. A Gaussian Process Perspective
  8. Connection to Regularized Regression.
  9. Decision Strategies: UCB and Thompson Sampling
  10. Practical Implementation
  11. Regret Analysis
  12. Assumptions
  13. High Probability Confidence Bound
  14. Regret Bounds via Effective Dimension
  15. Spectral Analysis of the Multi-User Kernel
  16. ...and 35 more sections

Key Result

Theorem 2.1

Let ${\mathcal{H}}_x$ be an RKHS of functions on ${\mathcal{D}}$ with kernel $K_x$. The vector space of function collections ${\mathcal{H}} := \{ (f_1, \dots, f_n) : f_u \in {\mathcal{H}}_x, \forall u \in {\mathcal{U}} \}$ equipped with the inner product is a Reproducing Kernel Hilbert Space of functions on ${\mathcal{U}} \times {\mathcal{D}}$. The associated squared RKHS norm is precisely the pe

Figures (6)

  • Figure 1: Rank Collapse: (a) Comparing the growth of the actual information gain $\gamma_T$ vs $n$ in i.i.d. design (red) versus the two bounds \ref{['eq:info:gain:operator:bound']} (blue; crude) and \ref{['eq:regular:design:info:gain']} (green; nearly exact) in a complete graph. The kernel is $\exp(-\|x-y\|^2/2)$, $u_i \sim \text{Unif}([n])$ and $x_i \sim \text{Unif}[0,1]^d$ where $d = 5$. Panels (b) and (c) show the growth of $\tilde{d}$ vs. $(\log T)^d$ under empty and complete graphs, respectively. Note that under the complete graph, $\tilde{d}$ slightly decreases as $n$ increases.
  • Figure 2: Cumulative Regret under Linear-GOB regime. From left to right are tasks of easy level, medium level, to hard level.
  • Figure 3: Cumulative Regret under Laplacian–Kernel regime using GP draw. From left to right are tasks of easy level, medium level, to hard level.
  • Figure 4: Cumulative Regret under Laplacian–Kernel regime using representer draw. From left to right are tasks of easy level, medium level, to hard level.
  • Figure 5: Comparison of the choice of user-similarity kernel for COOP-KernelUCB.
  • ...and 1 more figures

Theorems & Definitions (24)

  • Theorem 2.1: Multi-user Kernel
  • Remark 1
  • Theorem 4.1: Confidence Bound
  • Definition 4.1: Effective Dimension
  • Theorem 4.2: Regret Bound of LK-GP-UCB
  • Theorem 4.3: Regret Bound of LK-GP-TS
  • Proposition 4.1
  • proof
  • proof
  • Lemma C.1: Concentrations for TS
  • ...and 14 more