Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentially Private Linear Bandits with Partial Distributed Feedback

Fengjiao Li, Xingyu Zhou, Bo Ji

TL;DR

A unified algorithmic learning framework, called differentially private distributed phased elimination (DP-DPE), which can be naturally integrated with popular differential privacy (DP) models (including central DP, local DP, and shuffle DP), and it is proved that DP- DPE achieves both sublinear regret and sublinear communication cost.

Abstract

In this paper, we study the problem of global reward maximization with only partial distributed feedback. This problem is motivated by several real-world applications (e.g., cellular network configuration, dynamic pricing, and policy selection) where an action taken by a central entity influences a large population that contributes to the global reward. However, collecting such reward feedback from the entire population not only incurs a prohibitively high cost but often leads to privacy concerns. To tackle this problem, we consider differentially private distributed linear bandits, where only a subset of users from the population are selected (called clients) to participate in the learning process and the central server learns the global model from such partial feedback by iteratively aggregating these clients' local feedback in a differentially private fashion. We then propose a unified algorithmic learning framework, called differentially private distributed phased elimination (DP-DPE), which can be naturally integrated with popular differential privacy (DP) models (including central DP, local DP, and shuffle DP). Furthermore, we prove that DP-DPE achieves both sublinear regret and sublinear communication cost. Interestingly, DP-DPE also achieves privacy protection ``for free'' in the sense that the additional cost due to privacy guarantees is a lower-order additive term. In addition, as a by-product of our techniques, the same results of ``free" privacy can also be achieved for the standard differentially private linear bandits. Finally, we conduct simulations to corroborate our theoretical results and demonstrate the effectiveness of DP-DPE.

Differentially Private Linear Bandits with Partial Distributed Feedback

TL;DR

A unified algorithmic learning framework, called differentially private distributed phased elimination (DP-DPE), which can be naturally integrated with popular differential privacy (DP) models (including central DP, local DP, and shuffle DP), and it is proved that DP- DPE achieves both sublinear regret and sublinear communication cost.

Abstract

In this paper, we study the problem of global reward maximization with only partial distributed feedback. This problem is motivated by several real-world applications (e.g., cellular network configuration, dynamic pricing, and policy selection) where an action taken by a central entity influences a large population that contributes to the global reward. However, collecting such reward feedback from the entire population not only incurs a prohibitively high cost but often leads to privacy concerns. To tackle this problem, we consider differentially private distributed linear bandits, where only a subset of users from the population are selected (called clients) to participate in the learning process and the central server learns the global model from such partial feedback by iteratively aggregating these clients' local feedback in a differentially private fashion. We then propose a unified algorithmic learning framework, called differentially private distributed phased elimination (DP-DPE), which can be naturally integrated with popular differential privacy (DP) models (including central DP, local DP, and shuffle DP). Furthermore, we prove that DP-DPE achieves both sublinear regret and sublinear communication cost. Interestingly, DP-DPE also achieves privacy protection ``for free'' in the sense that the additional cost due to privacy guarantees is a lower-order additive term. In addition, as a by-product of our techniques, the same results of ``free" privacy can also be achieved for the standard differentially private linear bandits. Finally, we conduct simulations to corroborate our theoretical results and demonstrate the effectiveness of DP-DPE.
Paper Structure (28 sections, 11 theorems, 74 equations, 3 figures, 4 tables, 2 algorithms)

This paper contains 28 sections, 11 theorems, 74 equations, 3 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Global Reward Maximization with Partial Feedback
  4. Differentially Private Distributed Linear Bandits
  5. Algorithm Design
  6. Key Challenges
  7. Differentially Private Distributed Phased Elimination (DP-DPE)
  8. DP-DPE under Different DP Models
  9. DP-DPE under the Central DP Model
  10. DP-DPE under the Local DP Model
  11. DP-DPE under the Shuffle DP Model
  12. Main Results
  13. Numerical Results
  14. Discussion
  15. On Achieving Privacy "for Free"
  16. ...and 13 more sections

Key Result

Theorem 4.2

The DP-DPE instantiation using the Privatizer in Eq. eq:privatizer_cdp_app with $\sigma_{nc} = \frac{2B\sqrt{2s_l\ln(1.25/\delta)}}{\varepsilon |U_l|}$ guarantees $(\varepsilon, \delta)$-DP.

Figures (3)

  • Figure 1: Cellular network configuration: a motivating application of global reward maximization with partial feedback in a linear bandit setting.
  • Figure 2: Performance evaluation of DP-DPE. The shaded area indicates the standard deviation. (a) Final cumulative regret vs. the privacy budget $\varepsilon$. (b) Per-round regret vs. time with privacy parameters $\varepsilon=10$ and $\delta=0.1$. (c) Comparison between two non-private algorithms. Here, we choose the number of clients in DPE-FixedU to be $U=97$ based on the calculation.
  • Figure 3: LinUCB vs. PE vs DPE with different values of $\alpha$.

Theorems & Definitions (33)

  • Remark 3.1
  • Remark 3.2
  • Definition 4.1
  • Theorem 4.2
  • Definition 4.3
  • Theorem 4.4
  • Definition 4.5
  • Theorem 4.6
  • Theorem 5.1: DPE
  • proof : Proof sketch
  • ...and 23 more