Optimal Regret of Bernoulli Bandits under Global Differential Privacy

TL;DR

The paper delivers matching upper and lower regret bounds for Bernoulli bandits under ε-global DP by introducing a new information-theoretic quantity d_ε that smoothly interpolates KL and TV distances. It proves a tighter lower bound and designs two asymptotically optimal algorithms, DP-KLUCB and DP-IMED, that avoid forgetting past rewards and rely on a novel DP concentration inequality coupling Bernoulli rewards with Laplace noise. The proposed methods achieve regret matching the lower bound up to a multiplicative factor α arbitrarily close to 1, demonstrating that forgetting is not necessary for optimal DP bandit performance. Extensive experiments validate the theoretical gains over prior DP bandit algorithms. The work also contributes a DP version of a Chernoff-type bound and clarifies adaptive continual release perspectives in the DP bandit setting, with potential extensions to broader distribution families.

Abstract

As sequential learning algorithms are increasingly applied to real life, ensuring data privacy while maintaining their utilities emerges as a timely question. In this context, regret minimisation in stochastic bandits under $ε$-global Differential Privacy (DP) has been widely studied. Unlike bandits without DP, there is a significant gap between the best-known regret lower and upper bound in this setting, though they "match" in order. Thus, we revisit the regret lower and upper bounds of $ε$-global DP algorithms for Bernoulli bandits and improve both. First, we prove a tighter regret lower bound involving a novel information-theoretic quantity characterising the hardness of $ε$-global DP in stochastic bandits. Our lower bound strictly improves on the existing ones across all $ε$ values. Then, we choose two asymptotically optimal bandit algorithms, i.e. DP-KLUCB and DP-IMED, and propose their DP versions using a unified blueprint, i.e., (a) running in arm-dependent phases, and (b) adding Laplace noise to achieve privacy. For Bernoulli bandits, we analyse the regrets of these algorithms and show that their regrets asymptotically match our lower bound up to a constant arbitrary close to 1. This refutes the conjecture that forgetting past rewards is necessary to design optimal bandit algorithms under global DP. At the core of our algorithms lies a new concentration inequality for sums of Bernoulli variables under Laplace mechanism, which is a new DP version of the Chernoff bound. This result is universally useful as the DP literature commonly treats the concentrations of Laplace noise and random variables separately, while we couple them to yield a tighter bound.

Figures (8)

• Figure 1: Evolution of the regret over time for $\mathsf{DP\text{-}SE}$, AdaP-KLUCB, Lazy-DP-TS, $\mathsf{DP\text{-}KLUCB}$, and $\mathsf{DP\text{-}IMED}$ for $\epsilon = 0.25$, and Bernoulli bandits $\mu_1$ (left) and $\mu_2$ (right).
• Figure 2: Interactive protocol in the adaptive continual release model between a policy $\pi$ and a reward-feeding adversary $\mathcal{A}$. The protocol in Figure (a) is run with $b=L$, while the protocol in Figure (b) is run with $b=L$. The framed part corresponds to the reward observed by the policy.
• Figure 3: Different reward representations for $T = 3$ and $K = 2$. The highlighted rewards are the rewards observed by the policy for the trajectory $(a_1, a_2, a_3) = (1,2,1)$
• Figure 4: Evolution of regret over time for $\mu_1$ for different budgets $\epsilon$.
• Figure 5: Evolution of regret over time for $\mu_2$ for different budgets $\epsilon$.

Theorems & Definitions (33)

• Definition 1: Policy
• Definition 2: $\epsilon$-DP Dwork_Calibration
• Definition 3: Table DP and View DP azizeconcentrated
• Proposition 4: $\epsilon$-global DP, Proposition 1 in azizeconcentrated
• Theorem 5: Regret lower bound under $\epsilon$-global DP
• Definition 6: Maximal Couplings
• Proposition 7: Concentration Bound of Private Mean
• Proposition 8: Privacy analysis
• Theorem 9: Regret upper bound of $\mathsf{DP\text{-}IMED}$ and $\mathsf{DP\text{-}KLUCB}$
• Definition 10: Reward-Feeding Adversary