Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faster Rates for Private Adversarial Bandits

Hilal Asi, Vinod Raman, Kunal Talwar

TL;DR

The paper advances private online learning by introducing a simple batched-noise conversion that privatizes any non-private bandit algorithm, achieving sublinear regret under central DP with rates scaling as $O\left(\frac{\sqrt{KT}}{\sqrt{\varepsilon}}\right)$. It extends the approach to bandits with expert advice, delivering three DP guarantees whose regret scales with $N$, $K$, $T$, and $\varepsilon$ in regime-appropriate ways, including $O\left(\frac{\sqrt{NT}}{\sqrt{\varepsilon}}\right)$, $O\left(\frac{\sqrt{KT\log(N)}\log(KT)}{\varepsilon}\right)$, and $\tilde{O}\left(\frac{N^{1/6}K^{1/2}T^{2/3}\log(NT)}{\varepsilon^{1/3}} + \frac{N^{1/2} \log(NT)}{\varepsilon}\right)$. By leveraging Laplace noise on batched losses and standard DP composition, the authors obtain sublinear guarantees even for relatively large privacy budgets and establish a separation between central and local DP in these settings. The work also identifies fundamental barriers to achieving faster private rates for adversarial bandits, including hardness results for privatizing EXP3 and lower bounds for broad algorithm classes, clarifying the landscape of what is achievable in private adversarial online learning.

Abstract

We design new differentially private algorithms for the problems of adversarial bandits and bandits with expert advice. For adversarial bandits, we give a simple and efficient conversion of any non-private bandit algorithm to a private bandit algorithm. Instantiating our conversion with existing non-private bandit algorithms gives a regret upper bound of $O\left(\frac{\sqrt{KT}}{\sqrtε}\right)$, improving upon the existing upper bound $O\left(\frac{\sqrt{KT \log(KT)}}ε\right)$ for all $ε\leq 1$. In particular, our algorithms allow for sublinear expected regret even when $ε\leq \frac{1}{\sqrt{T}}$, establishing the first known separation between central and local differential privacy for this problem. For bandits with expert advice, we give the first differentially private algorithms, with expected regret $O\left(\frac{\sqrt{NT}}{\sqrtε}\right), O\left(\frac{\sqrt{KT\log(N)}\log(KT)}ε\right)$, and $\tilde{O}\left(\frac{N^{1/6}K^{1/2}T^{2/3}\log(NT)}{ε^{1/3}} + \frac{N^{1/2}\log(NT)}ε\right)$, where $K$ and $N$ are the number of actions and experts respectively. These rates allow us to get sublinear regret for different combinations of small and large $K, N$ and $ε.$

Faster Rates for Private Adversarial Bandits

TL;DR

The paper advances private online learning by introducing a simple batched-noise conversion that privatizes any non-private bandit algorithm, achieving sublinear regret under central DP with rates scaling as . It extends the approach to bandits with expert advice, delivering three DP guarantees whose regret scales with , , , and in regime-appropriate ways, including , , and . By leveraging Laplace noise on batched losses and standard DP composition, the authors obtain sublinear guarantees even for relatively large privacy budgets and establish a separation between central and local DP in these settings. The work also identifies fundamental barriers to achieving faster private rates for adversarial bandits, including hardness results for privatizing EXP3 and lower bounds for broad algorithm classes, clarifying the landscape of what is achievable in private adversarial online learning.

Abstract

We design new differentially private algorithms for the problems of adversarial bandits and bandits with expert advice. For adversarial bandits, we give a simple and efficient conversion of any non-private bandit algorithm to a private bandit algorithm. Instantiating our conversion with existing non-private bandit algorithms gives a regret upper bound of , improving upon the existing upper bound for all . In particular, our algorithms allow for sublinear expected regret even when , establishing the first known separation between central and local differential privacy for this problem. For bandits with expert advice, we give the first differentially private algorithms, with expected regret , and , where and are the number of actions and experts respectively. These rates allow us to get sublinear regret for different combinations of small and large and

Paper Structure

This paper contains 30 sections, 26 theorems, 166 equations, 1 figure, 2 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminaries
  4. Notation
  5. The Adversarial Bandit Problem
  6. The Bandits with Expert Advice Problem
  7. Differential Privacy
  8. Faster Rates for Private Adversarial Bandits
  9. Proof of Theorem \ref{['thm:banditconv']}
  10. Upper bounds for Bandits with Expert Advice
  11. Barriers to Even Faster Rates for Private Adversarial Bandits
  12. On the Hardness of Privatizing EXP3
  13. Limits of a Class of Adversarial Bandit Algorithms
  14. Privacy properties and privacy-preserving mechanisms
  15. Helper Lemmas
  16. ...and 15 more sections

Key Result

Theorem 1

Let $\mathcal{B}$ be any bandit algorithm. Then, for every $\tau \geq 1$ and $\varepsilon \leq 1$, there exists an $\varepsilon$-differentially private bandit algorithm $\mathcal{A}_{\tau}$ such that In particular, picking $\tau = \lceil\frac{1}{\varepsilon}\rceil$ means that there exists a $\varepsilon$-differentially private bandit algorithm $\mathcal{A}$ such that

Figures (1)

  • Figure 1: Probabilities on action $2$ assigned by EXP3 when run with $\gamma = \eta = 0.0001$, and $T = 100 \cdot \frac{1}{\eta}$ on datasets $\ell_{1:T}$ and $\ell^{\prime}_{1:T}$.

Theorems & Definitions (50)

  • Definition 1: Bandit Algorithm
  • Definition 2: Bandits with Expert Advice Algorithm
  • Definition 3: $(\varepsilon, \delta)$-Differential Privacy in Adversarial Bandits dwork2014algorithmic
  • Definition 4: $(\varepsilon, \delta)$-Differential Privacy in Bandits with Expert Advice dwork2014algorithmic
  • Theorem 1: Generic Conversion
  • Corollary 2: FTRL Conversion
  • Lemma 3: Privacy guarantee
  • proof
  • Lemma 4: Utility guarantee
  • Theorem 5: Theorem 2 in arora2012online
  • ...and 40 more