Rényi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by Iteration

TL;DR

This work extends privacy guarantees beyond differential privacy by introducing Rényi Pufferfish privacy (RPP), a flexible framework that handles correlated data and diverse adversarial priors. It develops a General Wasserstein Mechanism (GWM) for additive-noise mechanisms and then enhances utility through δ-approximation (GAWM) and p-Wasserstein shifts (DAGWM), linking to distribution privacy. It further establishes privacy amplification by iteration (PABI) for RPP, enabling private iterative algorithms for convex optimization without full composition. Collectively, the framework yields stronger utility than traditional group DP in several settings and provides practical tools for privacy-preserving machine learning on correlated data and diffusion-like processes.

Abstract

Pufferfish privacy is a flexible generalization of differential privacy that allows to model arbitrary secrets and adversary's prior knowledge about the data. Unfortunately, designing general and tractable Pufferfish mechanisms that do not compromise utility is challenging. Furthermore, this framework does not provide the composition guarantees needed for a direct use in iterative machine learning algorithms. To mitigate these issues, we introduce a Rényi divergence-based variant of Pufferfish and show that it allows us to extend the applicability of the Pufferfish framework. We first generalize the Wasserstein mechanism to cover a wide range of noise distributions and introduce several ways to improve its utility. We also derive stronger guarantees against out-of-distribution adversaries. Finally, as an alternative to composition, we prove privacy amplification results for contractive noisy iterations and showcase the first use of Pufferfish in private convex optimization. A common ingredient underlying our results is the use and extension of shift reduction lemmas.

Figures (3)

• Figure 1: Relations between the mechanisms and privacy notions studied in the paper. The values on the top of the graph represent the value $\varepsilon$ of the privacy budget guaranteed by the mechanisms. $\Delta_{G}$ corresponds to the sensitivity of the GWM (Section \ref{['sectionwassersteinmechanism']}), and $\Delta_{G,\delta}$ corresponds to the sensitivity of the GAWM (Section \ref{['sectionapproxwasserstein']}), $\Delta_G^{\zeta,1,\alpha}$ corresponds to the sensitivity of the DAGWM (Section \ref{['sectionrelaxWasserstein']}). $\Delta_{\text{GROUP}}$ corresponds to the sensitivity of mechanisms in the group privacy framework. The plain arrows indicate the privacy guarantees offered by the mechanisms. The dashed arrows compare the privacy budget offered by the mechanisms. The implication arrows illustrate the relations between the different frameworks.
• Figure 5: Privacy loss as a function of the number of iterations for the following values of $\rho_t$: $0 \text{ (DP)}, 0.1,0.2,0.8 \text{ and } 1 \text{ (Group DP)}$.
• Figure 6: Privacy loss as a function of the number of iterations for the following values of $\rho_t$ : $0 \text{ (DP)}, 0.1,1/t \text{ and }1/t^2$.

Theorems & Definitions (99)

• Definition 2.1: Pufferfish privacy, PP Kifer2014Ding2022
• Definition 2.2: Rényi Pufferfish privacy, RPP
• Proposition 2.1: Post-processing
• Proposition 2.2: RPP implies PP
• Definition 3.1: Couplings
• Definition 3.2: $\infty$-Wasserstein distance
• Lemma 3.1: Shift reduction Feldman2018
• Theorem 3.3: General Wasserstein mechanism, GWM
• Corollary 3.1: Privacy guarantees for usual noise distributions
• Proposition 3.1: Utility of the GWM, informal