Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Purifying Approximate Differential Privacy with Randomized Post-processing

Yingyu Lin, Erchi Wang, Yi-An Ma, Yu-Xiang Wang

TL;DR

This work introduces purification, a framework that converts $(\varepsilon,\delta)$-DP mechanisms into $(\varepsilon+\varepsilon')$-pure DP mechanisms via randomized post-processing and calibrated Laplace noise, under domain-geometry assumptions. The approach relies on a TV-to-$W_{\infty}$ distance conversion and a uniform-mixing step to enable pure-DP guarantees with near-optimal utility across tasks such as DP-ERM, PTR, mode release, and query release. The authors provide concrete purified algorithms (DP-SGD, DP-Frank-Wolfe, PTR purification, Pure DP mode release, AdaSSP-based regression, and MWEM-based query release) with matching or near-best-known pure-DP rates, and show how purification can be leveraged to derive lower bounds for approximate DP via contrapositive arguments. Overall, purification offers a practical blueprint for designing pure-DP algorithms by first leveraging approximate-DP routines and then applying structured post-processing to erase the delta while preserving utility. The framework broadens the toolkit for pure-DP algorithm design and provides a principled way to translate established approximate-DP methods into strong, zero-delta privacy guarantees with quantifiable utility.

Abstract

We propose a framework to convert $(\varepsilon, δ)$-approximate Differential Privacy (DP) mechanisms into $(\varepsilon', 0)$-pure DP mechanisms under certain conditions, a process we call ``purification.'' This algorithmic technique leverages randomized post-processing with calibrated noise to eliminate the $δ$ parameter while achieving near-optimal privacy-utility tradeoff for pure DP. It enables a new design strategy for pure DP algorithms: first run an approximate DP algorithm with certain conditions, and then purify. This approach allows one to leverage techniques such as strong composition and propose-test-release that require $δ>0$ in designing pure-DP methods with $δ=0$. We apply this framework in various settings, including Differentially Private Empirical Risk Minimization (DP-ERM), stability-based release, and query release tasks. To the best of our knowledge, this is the first work with a statistically and computationally efficient reduction from approximate DP to pure DP. Finally, we illustrate the use of this reduction for proving lower bounds under approximate DP constraints with explicit dependence in $δ$, avoiding the sophisticated fingerprinting code construction.

Purifying Approximate Differential Privacy with Randomized Post-processing

TL;DR

This work introduces purification, a framework that converts -DP mechanisms into -pure DP mechanisms via randomized post-processing and calibrated Laplace noise, under domain-geometry assumptions. The approach relies on a TV-to- distance conversion and a uniform-mixing step to enable pure-DP guarantees with near-optimal utility across tasks such as DP-ERM, PTR, mode release, and query release. The authors provide concrete purified algorithms (DP-SGD, DP-Frank-Wolfe, PTR purification, Pure DP mode release, AdaSSP-based regression, and MWEM-based query release) with matching or near-best-known pure-DP rates, and show how purification can be leveraged to derive lower bounds for approximate DP via contrapositive arguments. Overall, purification offers a practical blueprint for designing pure-DP algorithms by first leveraging approximate-DP routines and then applying structured post-processing to erase the delta while preserving utility. The framework broadens the toolkit for pure-DP algorithm design and provides a principled way to translate established approximate-DP methods into strong, zero-delta privacy guarantees with quantifiable utility.

Abstract

We propose a framework to convert -approximate Differential Privacy (DP) mechanisms into -pure DP mechanisms under certain conditions, a process we call ``purification.'' This algorithmic technique leverages randomized post-processing with calibrated noise to eliminate the parameter while achieving near-optimal privacy-utility tradeoff for pure DP. It enables a new design strategy for pure DP algorithms: first run an approximate DP algorithm with certain conditions, and then purify. This approach allows one to leverage techniques such as strong composition and propose-test-release that require in designing pure-DP methods with . We apply this framework in various settings, including Differentially Private Empirical Risk Minimization (DP-ERM), stability-based release, and query release tasks. To the best of our knowledge, this is the first work with a statistically and computationally efficient reduction from approximate DP to pure DP. Finally, we illustrate the use of this reduction for proving lower bounds under approximate DP constraints with explicit dependence in , avoiding the sophisticated fingerprinting code construction.

Paper Structure

This paper contains 62 sections, 44 theorems, 66 equations, 3 figures, 2 tables, 15 algorithms.

Table of Contents

  1. Introduction
  2. Summary of the Results
  3. Related Work
  4. Algorithm: Converting Approximate DP to Pure DP
  5. Technical Lemma: from TV distance to $\infty$-Wasserstein distance
  6. Empirical Risk Minimization with Pure Differential Privacy
  7. Purified DP Stochastic Gradient Descent
  8. Purified DP Frank-Wolfe Algorithm
  9. Pure DP Data-dependent Mechanisms
  10. Propose-Test-Release with Pure Differential Privacy
  11. Pure DP Mode Release
  12. Pure DP Linear Regression
  13. Pure DP Query Release
  14. Purification as a Tool for Proving Lower Bounds
  15. Conclusion and Limitation
  16. ...and 47 more sections

Key Result

Theorem 1

Define $x_{\text{pure}}, x_{\text{apx}}$ as in Algorithm alg:main under Assumption assump:domain. The output of Algorithm alg:main satisfies $(\varepsilon+\varepsilon')$-DP with utility guarantee

Figures (3)

  • Figure 1: Illustration of Example \ref{['ex:tightness']} on the metric space $(\mathbb{R}^2, \|\cdot\|_2)$
  • Figure 2: Trade-off functions for $(\varepsilon,\delta)$-DP, $(\varepsilon,0)$-DP, and $(\varepsilon+\varepsilon',0)$-DP. Our method provides a solution to post-process the $(\varepsilon,\delta)$-DP distribution pair (in blue) to the $(\varepsilon+\varepsilon',0)$-DP pair (in red).
  • Figure 3: Flowchart illustrating the proof sketch of Theorem \ref{['thm:pure_main']} and the intuition behind Algorithm \ref{['alg:main']}. The notation $D_\infty\leq \varepsilon$ is an abbreviation for the pair of inequalities $D_\infty(\mu\|\nu)\leq \varepsilon$ and $D_\infty(\nu\|\mu)\leq \varepsilon$, where $\mu$ and $\nu$ correspond to the two end nodes of the respective edges (e.g., $P$ and $P'$). The symbol $\omega \mathrm{Unif}$ represents a mixture with the uniform distribution (Algorithm \ref{['alg:main']}, Line \ref{['step:unif_1']}), where $\mathcal{U}(\cdot) = (1-\omega)\cdot + \omega\mathrm{Unif}(\Theta)$. The notation $*\text{Lap}$ refers to the convolution with the Laplace distribution, as in Algorithm \ref{['alg:main']}, Line \ref{['step:laplace']}.

Theorems & Definitions (63)

  • Definition 1: Differential privacy dwork2006calibratingdwork2014algorithmic
  • Theorem 1
  • Remark 2
  • Remark 3
  • Corollary 4: Parameters of Algorithm \ref{['alg:main']} for DP-SGD
  • Theorem 2
  • Definition 5
  • Lemma 6: Converting $d_\mathrm{{TV}}$ to $W_\infty$
  • Example 8: Tightness of the Conversion
  • Theorem 3: Utility, privacy, and runtime for purified DP-SGD
  • ...and 53 more