Are We There Yet? Timing and Floating-Point Attacks on Differential Privacy Systems

TL;DR

This work shows that common DP noise implementations are vulnerable to two practical side channels: floating-point representation errors in Gaussian sampling and timing leaks in discrete distribution samplers. It develops FP attacks that adapt Mironov’s Laplace attack to Gaussian samplers (Polar and Ziggurat) and demonstrates feasibility across NumPy, PyTorch, and Go implementations, including DP-SGD in Opacus. It also reveals timing side-channels in discrete Laplace and Gaussian samplers (and Google's approximate Laplace), showing that DP guarantees can be violated in end-to-end systems. The paper proposes mitigations such as constant-time sampling, batching/offline sampling, and cautious use of discretized variants, highlighting the ongoing need to align practical DP deployments with their theoretical guarantees. Overall, the findings stress careful, implementation-level validation of DP systems to preserve privacy in real-world settings.

Abstract

Differential privacy is a de facto privacy framework that has seen adoption in practice via a number of mature software platforms. Implementation of differentially private (DP) mechanisms has to be done carefully to ensure end-to-end security guarantees. In this paper we study two implementation flaws in the noise generation commonly used in DP systems. First we examine the Gaussian mechanism's susceptibility to a floating-point representation attack. The premise of this first vulnerability is similar to the one carried out by Mironov in 2011 against the Laplace mechanism. Our experiments show attack's success against DP algorithms, including deep learning models trained using differentially-private stochastic gradient descent. In the second part of the paper we study discrete counterparts of the Laplace and Gaussian mechanisms that were previously proposed to alleviate the shortcomings of floating-point representation of real numbers. We show that such implementations unfortunately suffer from another side channel: a novel timing attack. An observer that can measure the time to draw (discrete) Laplace or Gaussian noise can predict the noise magnitude, which can then be used to recover sensitive attributes. This attack invalidates differential privacy guarantees of systems implementing such mechanisms. We demonstrate that several commonly used, state-of-the-art implementations of differential privacy are susceptible to these attacks. We report success rates up to 92.56% for floating-point attacks on DP-SGD, and up to 99.65% for end-to-end timing attacks on private sum protected with discrete Laplace. Finally, we evaluate and suggest partial mitigations.

Figures (11)

• Figure 1: Distribution and number of values amenable to a floating-point attack against a Gaussian implementation $\mathsf{Gauss\_polar}$ using $R=2^{10}$ and $\sigma=114$. Here the horizontal axis, $f(D) +s$, shows both values $f(D) + s_1$ and $f(D) + s_2$. For presentation purposes, rate of attackable floating-point values is averaged over $0.8$-wide intervals. Blue line indicates the rate of floating-point values that are in the support of $f(D) = 0$ and not in $f(D') = 1$. Gray line measures average number of times support for $f(D)$ was observed.
• Figure 2: FP attack success rate on private count where the count is protected with one of the three Gaussian samplers across $\epsilon \in (0,100]$ with fixed $\delta=10^{-5}$ and function sensitivity $\Delta=1$. Baseline (random) attack success is 50%.
• Figure 3: FP attack success rate against DP-SGD on MNIST model using the Opacus opacus. $\mathsf{DiffLabelCanary}$: attacker is distinguishing between a batch where all records have identical labels in the range $[1,9]$ and a batch that has same records as well as an image for label 0. $\mathsf{SimLabelCanary}$: batches where all records are from $[0,9]$ and differ in one random record. Since the baseline attack is 50%, the plot shows that FP attack is successful on $\mathsf{DiffLabelCanary}$ where the canary record comes from the same distribution as MNIST data but different from other records in the batch. Here, $\delta = 10^{-5}$.
• Figure 4: Average time in $\mu$s (with the distribution's 0.25 and 0.75 quantiles) to generate absolute noise using two implementations of discrete distributions: (a) Discrete Laplace and Gaussian from Implementation I discrete-imp (avg. over 10 million trials). (b) Discrete Laplace from Implementation II google-dp (avg. over 20 million trials). The plots show a linear relationship between the absolute noise value and time it takes to generate it. Gaussian I uses $\sigma=19$, Laplace I uses $\lambda=8$ and Laplace II uses $\lambda=\frac{8}{\ln{3}}$.
• Figure 5: Attack on private sum of the credit attribute of the German Credit Dataset german-credits, with Laplace II google-dp. The attack success rate under different privacy budget $\epsilon\in[1,10]$ and sensitivity $\Delta=5000$ (measured over 1 million trials for each $\epsilon$). Success means the private sum created from $D$ is successfully concluded to be in support of $D$ and not $D'$.