GPM: The Gaussian Pancake Mechanism for Planting Undetectable Backdoors in Differential Privacy

TL;DR

The Gaussian pancake mechanism (GPM), a novel mechanism that is computationally indistinguishable from the widely used Gaussian mechanism (GM), yet exhibits arbitrarily weaker statistical DP guarantees, enables a new class of backdoor attacks: by indistinguishably passing off as the authentic GM, GPM can covertly degrade statistical privacy.

Abstract

Differential privacy (DP) has become the gold standard for preserving individual privacy in data analysis. However, an implicit yet fundamental assumption underlying these rigorous privacy guarantees is the correct implementation and execution of DP mechanisms. Several incidents of unintended privacy loss have occurred due to numerical issues and inappropriate configurations of DP software, which have been successfully exploited in privacy attacks. To better understand the seriousness of defective DP software, we ask the following question: is it possible to elevate these passive defects into active privacy attacks while maintaining covertness? To address this question, we present the Gaussian pancake mechanism (GPM), a novel mechanism that is computationally indistinguishable from the widely used Gaussian mechanism (GM), yet exhibits arbitrarily weaker statistical DP guarantees. This unprecedented separation enables a new class of backdoor attacks: by indistinguishably passing off as the authentic GM, GPM can covertly degrade statistical privacy. Unlike the unintentional privacy loss caused by GM's numerical issues, GPM is an adversarial yet undetectable backdoor attack against data privacy. We formally prove GPM's covertness, characterize its statistical leakage, and demonstrate a concrete distinguishing attack that can achieve near-perfect success rates under suitable parameter choices, both theoretically and empirically. Our results underscore the importance of using transparent, open-source DP libraries and highlight the need for rigorous scrutiny and formal verification of DP implementations to prevent subtle, undetectable privacy compromises in real-world systems.

Figures (7)

• Figure 1: Probability densities of the Gaussian distribution and the hCLWE distribution in $d=2$. Note that distinguishing between the two distributions is assumed to be a hard problem for higher $d$, where $d$ can be viewed as an instantiation of the security parameter $\kappa$.
• Figure 2: Illustration of the Backdoor Attack Model: the supply-chain backdoor planter (SCBP) plants the backdoor by replacing GM with GPM. Consequently, the backdoor privacy attacker (BPA) gains information on $D$ using GPM's output $\mathbf{y}$ and knowledge of the secret backdoor key $\mathbf{w}$. However, against a regular privacy attacker (RPA) without knowledge of $\mathbf{w}$, the GPM output is computationally indistinguishable from GM, thus preserving the same privacy guarantees. For the same reason, neither the server nor other parties (e.g., downstream users) can detect the existence of the backdoor.
• Figure 3: When projected onto the secret direction $\mathbf{w}$, the probability densities of $\mathcal{M}_{\sigma, \mathbf{w}, \beta, \gamma}\left(D\right)$ and $\mathcal{M}_{\sigma, \mathbf{w}, \beta, \gamma}\left(D'\right)$ for neighbouring databases $D\simeq D'$ are concentrated in disjoint sets of intervals and are therefore well separated, breaking the original DP guarantee. In this example, $q\left(D\right)$ and $q\left(D'\right)$ differ by $2.4$ peak widths, such that $T = 2$ and $t = 0.4$.
• Figure 4: Comparison between GM and GPM's privacy costs at $t = 0.25$, as in Theorem \ref{['thm:gpm-dp-lb']}. Although computationally indistinguishable from GM, GPM's actual privacy costs exceed $\varepsilon=10^4$ even when $\delta = 0.1$, which is infeasible for practical applications. Note that the lower and upper bounds of GPM's privacy costs still change with respect to $\delta$, although at smaller relative scales.
• Figure 5: Distinguishing attack success rates on DP-hist. Success rates approach $1$ when $\beta < 10^{-3}$, and degrade significantly for $\beta \in \left\{10^{-1}, 10^{-2}\right\}$, especially for $\gamma = 2\sqrt{d}$. For larger $\beta$, increasing $\varepsilon^*$ slightly improves performance.

Theorems & Definitions (30)

• Definition 2.1: differential privacy DBLP:journals/fttcs/DworkR14DBLP:conf/tcc/DworkMNS06
• Definition 2.2: Gaussian mechanism
• Theorem 2.3
• Definition 2.4: discrete Gaussian mechanism dg
• Definition 2.5: hCLWE distribution clwe
• Lemma 2.6
• Theorem 2.7
• Definition 3.1: Gaussian pancake mechanism
• Example 3.3
• Theorem 3.4: Covertness of GPM backdoor