Sequentially Auditing Differential Privacy

TL;DR

This work develops a sequential, anytime-valid framework for auditing differential privacy guarantees of black-box mechanisms using kernel-based Maximum Mean Discrepancy (MMD) and e-value concepts. The authors derive a general MMD-based one-sided sequential test with provable Type I error control and exponential growth under the alternative, and provide a practical instantiation using Online Newton Step and Online Gradient Ascent to learn the witness and tuning parameter. They prove a tighter MMD-DP bound and demonstrate substantial improvements in sample efficiency, enabling DP violation detection with as few as a few hundred samples, compared to batch methods requiring tens to hundreds of thousands. Empirical results on additive-noise mechanisms and DP-SGD show rapid detection of privacy breaches and the ability to infer empirical privacy bounds during training, highlighting the method’s practical impact for rapid, resource-efficient privacy auditing and verification.

Abstract

We propose a practical sequential test for auditing differential privacy guarantees of black-box mechanisms. The test processes streams of mechanisms' outputs providing anytime-valid inference while controlling Type I error, overcoming the fixed sample size limitation of previous batch auditing methods. Experiments show this test detects violations with sample sizes that are orders of magnitude smaller than existing methods, reducing this number from 50K to a few hundred examples, across diverse realistic mechanisms. Notably, it identifies DP-SGD privacy violations in \textit{under} one training run, unlike prior methods needing full model training.

Figures (8)

• Figure 1: Sequential audit results for DP-SGD implementations during training under white-box access with canary gradient threat model over 5 independent runs. Private implementations (left) are correctly identified as satisfying the specified differential privacy guarantee, while non-private implementations (right) are successfully detected as privacy violations for non-trivial values of $\varepsilon$.
• Figure 2: Sequential audit results using e-process based testing (\ref{['alg:alternative-seq-dp-testing']}) for DP-SGD implementations during training. White-box access with canary gradient threat model over 5 independent runs as in \ref{['sec:dpsgd-audit']}. The e-process approach demonstrates faster detection rates compared to \ref{['alg:seq-dp-testing']} while maintaining accurate identification of privacy-preserving mechanisms.
• Figure 3: Type I error control in sequential testing. The auditing process for 100 simulations comparing two identical 2-dimensional uniform distributions on $[0,1]^2$. All trajectories remain below the rejection threshold (horizontal dashed line at $1/\alpha$), confirming proper Type I error control at the $\alpha = 0.05$ significance level.
• Figure 4: Detection power under alternative hypothesis. We calculate the auditing process for 100 simulations comparing a uniform distribution on $[0,1]^2$ against a perturbed uniform distribution. The exponential growth of the auditing process leads to rejection of the null hypothesis after 108 observations on average.
• Figure 5: Power analysis of sequential MMD test across dimensions and separation levels. The plot shows rejection rates over 20 simulations comparing $\mathcal{N}(0,I_d)$ against $\mathcal{N}(\mu,I_d)$ with varying dimensionality $d$ and mean separation $\mu$. Type I error is controlled when $||\mu||=0$.

Theorems & Definitions (23)

• Definition 2.1
• Definition 2.2: Approximate Differential Privacy dwork2006
• Definition 2.3
• Definition 2.4: Nonnegative supermartingale
• Theorem 2.1: Ville's inequality ville1939etude
• Definition 3.1
• Theorem 3.1
• Definition 3.2
• Theorem 3.2
• Theorem 3.3: Statistical properties of Algorithm \ref{['alg:seq-dp-testing']}