Total Variation Meets Differential Privacy

TL;DR

The paper introduces a refinement of approximate differential privacy by tracking the total variation of a privacy mechanism (η-TV), and proves an exact, tight k-fold adaptive composition bound under (ε,δ)-DP and η-TV. It constructs a dominating mechanism to realize the full DP-η-TV privacy region and uses Blackwell’s theorem to establish tightness, while also showing that η-TV is closed under subsampling and that TV-aware analyses yield tighter privacy-utility guarantees for learning algorithms such as DP-SGD. The work further connects TV with local differential privacy via contraction coefficients, derives exact TV expressions for Laplace, Gaussian, and staircase mechanisms, and demonstrates practical implications for membership inference attacks and privacy-utility tradeoffs. Overall, TV-aware DP provides more precise privacy accounting, sharper composition behavior, and actionable guidance for privacy-preserving data analysis and learning.

Abstract

The framework of approximate differential privacy is considered, and augmented by leveraging the notion of ``the total variation of a (privacy-preserving) mechanism'' (denoted by $η$-TV). With this refinement, an exact composition result is derived, and shown to be significantly tighter than the optimal bounds for differential privacy (which do not consider the total variation). Furthermore, it is shown that $(\varepsilon,δ)$-DP with $η$-TV is closed under subsampling. The induced total variation of commonly used mechanisms are computed. Moreover, the notion of total variation of a mechanism is studied in the local privacy setting and privacy-utility tradeoffs are investigated. In particular, total variation distance and KL divergence are considered as utility functions and studied through the lens of contraction coefficients. Finally, the results are compared and connected to the locally differentially private setting.

Figures (7)

• Figure 1: ROC corresponding to $(\varepsilon,\delta)$-DP.
• Figure 2: ROC corresponding to $(\varepsilon,\delta)$-DP $\eta$-TV.
• Figure 3: The Laplace mechanism's exact tradeoff function is shown in green (for $\varepsilon=1$). The blue lines correspond to the differential privacy constraints. The red line corresponds to $\beta_{\mathrm{I}} = 1 - \eta_{\mathrm{Lap}} - \beta_{\mathrm{II}}$ where $\eta_{\mathrm{Lap}} = 1-e^{-\varepsilon/2}$ is the total variation of the Laplace mechanism.
• Figure 4: The induced tradeoff functions of the dominating mechanisms for $\varepsilon = 1$, $\delta = 0$, and $\alpha=0.3$. The green line shows the tradeoff function of $\varepsilon$-DP (given by equation \ref{['eq:composition-kairouz']}). The blue and red lines plot the tradeoff given by Theorem \ref{['thm:CompositionTVApproximate']}, where they correspond, respectively, to odd (same parity as $k=5$) and even choices of $j$ in equation \ref{['eq:thm-compositionTV']}.
• Figure 5: Comparison of the resulting composition bounds: dataset size $n = 60,000$, batch size $m = 256$, learning rate $= 0.25$, clipping threshold $= 1.5$. Noisy-SGD algorithm running for 15 epochs. Moments accountant refers to the method developed by Abadi et al.abadi:16 which yields $(1.19,10^{-5})$-DP; Asymptotic SGD corresponds to the asymptotic result of GaussianDP (both of these curves were retrieved from GaussianDP); Theorem 2 is applied for all $\varepsilon \in [0.5, 3.4]$ in steps of 0.1 while using $\eta$ of the mechanism; Kairouz et al.TheCompositionTheoremForDP is applied for the same values of $\varepsilon$.

Theorems & Definitions (14)

• Definition 1: Differential Privacy (DP) DworkDP
• Definition 2: ROC
• Definition 3: Total Variation
• Definition 4: Total Variation of a Mechanism
• Remark 1
• Remark 2
• Remark 3
• Definition 5: $f$-DP GaussianDP
• Remark 4
• Definition 6: Local Differential Privacy