Information Contraction under $(\varepsilon,δ)$-Differentially Private Mechanisms

TL;DR

This work addresses how information distinguishability contracts when data are processed by mechanisms satisfying $(\varepsilon,\delta)$-local differential privacy. It develops both linear and nonlinear strong data-processing inequalities (SDPIs) for hockey-stick divergences $E_\gamma$ and for $f$-divergences $D_f$, valid for all $(\varepsilon,\delta)$-LDP, thereby generalizing prior results that were limited to $\delta=0$. A key contribution is the introduction of $F_\gamma$ curves to characterize input-output tradeoffs and the analysis of sequential composition across multiple private channels, yielding tighter contraction bounds and applicability to broader privacy settings. The results recover the known $\delta=0$ bounds and improve upon them in the general $(\varepsilon,\delta)$ setting, with practical implications for privacy-aware statistical tasks and inference under LDP. The paper also outlines potential tightness regimes and notes a quantum extension in NGH2025nonHS as a direction for future work.

Abstract

The distinguishability quantified by information measures after being processed by a private mechanism has been a useful tool in studying various statistical and operational tasks while ensuring privacy. To this end, standard data-processing inequalities and strong data-processing inequalities (SDPI) are employed. Most of the previously known and even tight characterizations of contraction of information measures, including total variation distance, hockey-stick divergences, and $f$-divergences, are applicable for $(\varepsilon,0)$-local differential private (LDP) mechanisms. In this work, we derive both linear and non-linear strong data-processing inequalities for hockey-stick divergence and $f$-divergences that are valid for all $(\varepsilon,δ)$-LDP mechanisms even when $δ\neq 0$. Our results either generalize or improve the previously known bounds on the contraction of these distinguishability measures.

Figures (2)

• Figure 1: Comparison of information contraction inequalities: DPI refers to the standard data-processing inequality in \ref{['eq:DPI_E_gamma']}; Linear SDPI refers to \ref{['prop:contraction_coeff_upper_bound']}; and Non-Linear SDPI refers to \ref{['thm:non_linear_HS_div']}. In this example setting, we consider $\varepsilon= \ln(6), \gamma'=2.5 < e^\varepsilon, \delta=0.01$ for a mechanism $A$ satisfying $(\varepsilon,\delta)$-LDP. Each of these lines/curves shows the largest $E_{\gamma'}\!\left(A(P_X) \Vert A(Q_X)\right)$ value that can be reached for the input distinguishability $E_{\gamma'}(P_X \Vert Q_X) \in [0,1]$.
• Figure 2: Comparing LDP bounds on the relative entropy. Dashed lines represent Equation \ref{['Eq:dasgupta']} (in particular a lower bound on that since we chose $\lambda=m$) and solid lines our new bound in Equation \ref{['Eq:RE-LDP-bound']}. (a): Plot over $\lambda$, respectively $m$, for fixed $\epsilon=\{1,2,3\}, \delta=0.01, \tau=0.25$. (b): Plot over $\epsilon$ for fixed $\delta=\{0.1,0.2,0.3\}, \lambda=m=0.1, \tau=0.25$.

Theorems & Definitions (21)

• Definition 1: Local Differential Privacy
• Proposition 1: Proposition 2, zamanlooy2024mathrm
• Corollary 1
• proof
• Corollary 2
• proof
• Proposition 2: Linear SDPI
• proof
• Remark 1
• Theorem 1: Non-Linear SDPI