Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exactly Minimax-Optimal Locally Differentially Private Sampling

Hyun-Young Park, Shahab Asoodeh, Si-Hyeon Lee

TL;DR

This work defines the fundamental PUT of private sampling in the minimax sense, using the f-divergence between original and sampling distributions as the utility measure, and proposes sampling mechanisms that are universally optimal for all f-divergences.

Abstract

The sampling problem under local differential privacy has recently been studied with potential applications to generative models, but a fundamental analysis of its privacy-utility trade-off (PUT) remains incomplete. In this work, we define the fundamental PUT of private sampling in the minimax sense, using the f-divergence between original and sampling distributions as the utility measure. We characterize the exact PUT for both finite and continuous data spaces under some mild conditions on the data distributions, and propose sampling mechanisms that are universally optimal for all f-divergences. Our numerical experiments demonstrate the superiority of our mechanisms over baselines, in terms of theoretical utilities for finite data space and of empirical utilities for continuous data space.

Exactly Minimax-Optimal Locally Differentially Private Sampling

TL;DR

This work defines the fundamental PUT of private sampling in the minimax sense, using the f-divergence between original and sampling distributions as the utility measure, and proposes sampling mechanisms that are universally optimal for all f-divergences.

Abstract

The sampling problem under local differential privacy has recently been studied with potential applications to generative models, but a fundamental analysis of its privacy-utility trade-off (PUT) remains incomplete. In this work, we define the fundamental PUT of private sampling in the minimax sense, using the f-divergence between original and sampling distributions as the utility measure. We characterize the exact PUT for both finite and continuous data spaces under some mild conditions on the data distributions, and propose sampling mechanisms that are universally optimal for all f-divergences. Our numerical experiments demonstrate the superiority of our mechanisms over baselines, in terms of theoretical utilities for finite data space and of empirical utilities for continuous data space.

Paper Structure

This paper contains 46 sections, 15 theorems, 127 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Notations and preliminaries
  4. System model
  5. Related work
  6. Main Results
  7. Optimal private sampling over finite space
  8. Optimal private sampling over continuous space
  9. Proof sketch of the theorems
  10. Discussions on the Proposed Mechanism
  11. Effect of the $r_P$ approximation error
  12. Continuity of the proposed mechanism
  13. Numerical Results
  14. Comparison for finite data space
  15. Comparison for 1D Gaussian mixture
  16. ...and 31 more sections

Key Result

Theorem 3.1

For each $k \in \mathbb{N}$, $\epsilon>0$, and an $f$-divergence $D_f$, we have Moreover, the mechanism $\mathbf{Q}^*_{k,\epsilon}$ constructed as below satisfies $\epsilon$-LDP and is optimal for $(\mathcal{X} = [k], \tilde{\mathcal{P}}=\mathcal{P}([k]), \epsilon)$ under any $D_f$: where $r_P > 0$ is a constant depending on $P$ so that $\sum_{x=1}^{k} \mathbf{Q}^*_{k,\epsilon}(x|P)=1$. Furtherm

Figures (7)

  • Figure 1: Original Gaussian ring distribution and the sampling distributions of the baseline Husain20-LDPSampling and our proposed mechanism for privacy budget $\epsilon=0.5$. The implementation details are in Appendix \ref{['supp:numResultSetups']}.
  • Figure 2: A visualization of the mechanism $\mathbf{Q}^*_{k,\epsilon}$
  • Figure 3: Theoretical worst-case $f$-divergences of proposed and previously proposed baseline mechanisms (with uniform $Q_0$) over finite space ($k=10$) (Left: KL divergence, Center: Total variation distance, Right: Squared Hellinger distance)
  • Figure 4: Empirical worst-case $f$-divergences of proposed and baseline mechanisms over 100 experiments of 1D Gaussian mixture (Left: KL divergence, Center: Total variation distance, Right: Squared Hellinger distance)
  • Figure 5: Theoretical worst-case $f$-divergences of proposed and previously proposed baseline mechanisms (with uniform $Q_0$) over finite space ($k=5$)
  • ...and 2 more figures

Theorems & Definitions (20)

  • Definition 2.1
  • Theorem 3.1
  • Proposition 3.2
  • Theorem 3.3
  • Proposition 3.4
  • Remark 3.5
  • Proposition 4.1
  • Theorem B.1
  • Theorem B.2: Data-Processing Inequality
  • Definition C.1
  • ...and 10 more