Breaking the Communication-Privacy-Accuracy Tradeoff with $f$-Differential Privacy

TL;DR

This work tackles privacy-communication-accuracy tradeoffs in federated data analysis by analyzing discrete-valued local mechanisms under $f$-differential privacy. It derives closed-form $f$-DP guarantees for binomial noise and the binomial mechanism, establishing tight privacy bounds and enabling compression-aware privacy analysis. Building on this, the authors introduce a ternary stochastic compressor that exploits privacy amplification by sparsification, achieving the same mean-squared error (MSE) and communication cost with substantially stronger privacy guarantees in distributed mean estimation. The vector extension leverages GDP via a central limit theorem for $f$-DP and Kashin representations to preserve privacy in high dimensions, and experiments demonstrate improved privacy-accuracy-communication tradeoffs compared to state-of-the-art baselines like SQKR and Gaussian mechanisms. Overall, the work provides a principled framework and practical compressor design for privacy-preserving, communication-efficient federated data analytics, with broad applicability to discrete-valued mechanisms.

Abstract

We consider a federated data analytics problem in which a server coordinates the collaborative data analysis of multiple users with privacy concerns and limited communication capability. The commonly adopted compression schemes introduce information loss into local data while improving communication efficiency, and it remains an open problem whether such discrete-valued mechanisms provide any privacy protection. In this paper, we study the local differential privacy guarantees of discrete-valued mechanisms with finite output space through the lens of $f$-differential privacy (DP). More specifically, we advance the existing literature by deriving tight $f$-DP guarantees for a variety of discrete-valued mechanisms, including the binomial noise and the binomial mechanisms that are proposed for privacy preservation, and the sign-based methods that are proposed for data compression, in closed-form expressions. We further investigate the amplification in privacy by sparsification and propose a ternary stochastic compressor. By leveraging compression for privacy amplification, we improve the existing methods by removing the dependency of accuracy (in terms of mean square error) on communication cost in the popular use case of distributed mean estimation, therefore breaking the three-way tradeoff between privacy, communication, and accuracy. Finally, we discuss the Byzantine resilience of the proposed mechanism and its application in federated learning.

Figures (6)

• Figure 1: Impact of $M$ on Algorithm \ref{['AddBinomialMechanism']} with $l=8$.
• Figure 2: Impact of $M$ on Algorithm \ref{['BinomialMechanism']}.
• Figure 3: Sparsification improves privacy.
• Figure 4: For the left figure, we set $k=10$ and derive the corresponding variance for SQKR, based on which $A$ and $B$ for the ternary stochastic compressor are computed such that they have the same communication overhead and MSE in expectation. The middle and right figures show the tradeoff between $\mu$-GDP and MSE. For the middle figure, we set $\sigma \in \{\frac{2}{5},\frac{1}{2},\frac{2}{3},1,2,4,6,8,10\}$ for the Gaussian mechanism, given which $A$ and $B$ are computed such that $AB = c^{2}+\sigma^{2}$ and the sparsity ratio is $A/B$. For the right figure, we set $A \in \{5c,10c,20c,30c\}$ and $A/B \in \{0.2,0.4,0.6,0.8,1.0\}$, given which the corresponding $\sigma$'s are computed such that $AB = c^{2}+\sigma^{2}$.
• Figure 5: Dividing $\alpha$ into multiple segments for part (1).

Theorems & Definitions (46)

• Definition 1: $(\epsilon,\delta)$-DP dwork2006our
• Definition 2: tradeoff function dong2021gaussian
• Definition 3: $f$-DP dong2021gaussian
• Lemma 1
• Theorem 1
• Remark 1
• Theorem 2
• Remark 2: Comparison to chen2022poisson
• Example 1
• Definition 4: Two-Level Stochastic Compressor jin2020stochastic