Differentially Private Kernelized Contextual Bandits
Nikola Pavlovic, Sudeep Salgia, Qing Zhao
TL;DR
This work addresses contextual kernel bandits where the reward function resides in an RKHS and contexts are revealed over time. It introduces USCA, a privacy-preserving algorithm that uses data-independent uniform sampling during learning and a novel low-sensitivity estimator to achieve joint differential privacy with respect to contexts and rewards. The main result is a diminishing simple-regret bound of $\mathcal{O}\left(\sqrt{\frac{\gamma_T}{T}} + \frac{\gamma_T}{T\varepsilon}\right)$ for kernels with polynomial eigen-decay, matching non-private rates up to the privacy term and extending applicability beyond the Square Exponential family. The combination of a forward-private sampling strategy and a covariance-based estimator yields strong privacy guarantees (JDP) while maintaining near-optimal learning performance across a broad class of kernels, including Matérn kernels.
Abstract
We consider the problem of contextual kernel bandits with stochastic contexts, where the underlying reward function belongs to a known Reproducing Kernel Hilbert Space (RKHS). We study this problem under the additional constraint of joint differential privacy, where the agents needs to ensure that the sequence of query points is differentially private with respect to both the sequence of contexts and rewards. We propose a novel algorithm that improves upon the state of the art and achieves an error rate of $\mathcal{O}\left(\sqrt{\frac{γ_T}{T}} + \frac{γ_T}{T \varepsilon}\right)$ after $T$ queries for a large class of kernel families, where $γ_T$ represents the effective dimensionality of the kernel and $\varepsilon > 0$ is the privacy parameter. Our results are based on a novel estimator for the reward function that simultaneously enjoys high utility along with a low-sensitivity to observed rewards and contexts, which is crucial to obtain an order optimal learning performance with improved dependence on the privacy parameter.