Privacy Audit as Bits Transmission: (Im)possibilities for Audit by One Run

TL;DR

The paper tackles the challenge of empirically auditing differential privacy by reducing the problem to a bit transmission task over a noisy channel, enabling principled lower bounds on privacy guarantees.It introduces a universal information-theoretic framework (with components $(n,p,\mathcal{H},\mathcal{C}_{\mathcal{M}},\mathcal{D})$) and derives fundamental limits on information leakage via $f$-DP and related trade-off functions, unifying single-run and multi-run auditing approaches.A key contribution is a detailed analysis of when privacy-audit-by-one-run is possible or infeasible, plus an audit method that yields tighter empirical lower bounds and can detect flaws in private algorithms with significantly fewer observations.Experiments across Gaussian mechanisms, $\mu$-GDP, DP-SGD, and a case study on a buggy implementation demonstrate superior tightness and practical utility, offering concrete guidelines for conducting efficient and reliable privacy audits.

Abstract

Auditing algorithms' privacy typically involves simulating a game-based protocol that guesses which of two adjacent datasets was the original input. Traditional approaches require thousands of such simulations, leading to significant computational overhead. Recent methods propose single-run auditing of the target algorithm to address this, substantially reducing computational cost. However, these methods' general applicability and tightness in producing empirical privacy guarantees remain uncertain. This work studies such problems in detail. Our contributions are twofold: First, we introduce a unifying framework for privacy audits based on information-theoretic principles, modeling the audit as a bit transmission problem in a noisy channel. This formulation allows us to derive fundamental limits and develop an audit approach that yields tight privacy lower bounds for various DP protocols. Second, leveraging this framework, we demystify the method of privacy audit by one run, identifying the conditions under which single-run audits are feasible or infeasible. Our analysis provides general guidelines for conducting privacy audits and offers deeper insights into the privacy audit. Finally, through experiments, we demonstrate that our approach produces tighter privacy lower bounds on common differentially private mechanisms while requiring significantly fewer observations. We also provide a case study illustrating that our method successfully detects privacy violations in flawed implementations of private algorithms.

Figures (10)

• Figure 1: The universal framework for privacy audit. Each membership inference corresponds to recovering a bit. Execution of the targeted private algorithm $\mathcal{M}$ corresponds to the usage of a noisy channel for bits transmission. $\mathcal{C}_\mathcal{M}$ is the noisy channel where execution of $\mathcal{M}$ happens, and $\mathcal{D}$ is where the membership inference is launched. $\mathcal{H}$ is the dataset generator and $m$ is what can be observed by the adversary.
• Figure 2: Single-bit transmission, modeled as a binary channel. If input bit $b_i=0$, the channel flips the bit with probability $\alpha$, corresponding to a false positive rate; if $b_i=1$, the bit is flipped with probability $\beta$, which is the false negative rate. As governed by the trade-off function, $\beta\geq f(\alpha)$ must hold. if $\alpha=\beta$, the above channel is the well-known binary symmetric channel (BSC).
• Figure 3: The mutual information upper bound $u_f(p)$ for different trade-off functions. $\delta=10^{-5}$ for $(\varepsilon,\delta)$-DP.
• Figure 4: Illustration on how our advanced CI method works. The horizontal axis is different $p_{min}^e$ value we assume that we can achieve, and the vertical axis is the corresponding lower bound. Line marked as $p_{min}^e$ corresponds to lower bound derived based on average bit error $p_{min}^e$ can be achieved; the same is to $\bar{e}+v(p_{min}^e,n,\gamma)$. Hoeffding result is the simple CI result shown in Equation \ref{['equ:hoeffding_bound_interval']}. Regions on the left of the vertical black line are where we have contradictions.
• Figure 5: Diagram of the case where we have $n=2$ bits of transmission. When there is interference, output bit $\hat{b}_1$ also depends on input $b_2$, but $\hat{b}_1$ is only intended to recover $b_1$.

Theorems & Definitions (22)

• Definition 1: Differential Privacy DBLP:conf/tcc/DworkMNS06
• Definition 2: Trade-off function dong2019gaussian
• Definition 3: $f$-DP dong2019gaussian
• Definition 4: $\mu$-Gaussian DP ($\mu$-GDP) dong2019gaussian
• Theorem 1: $(\varepsilon,\delta)$-DP's testing region kairouz2015composition
• Definition 5: Privacy audit as bits transmission
• Definition 6
• Lemma 1: Mixture by convex combination only makes it more indistinguishable, proof in Appendix \ref{['app:proof_mix_become_harder']}
• Corollary 1: Recovering bits is hard, proof in Appendix \ref{['app:proof_marginal_bit_recover_hard']}
• Theorem 2: Mutual information upper bound for bits transmission, proof in Appendix \ref{['app:proof_mi_bound_bit_trans']}