Muffliato: Peer-to-Peer Privacy Amplification for Decentralized Optimization and Averaging

TL;DR

This work targets privacy in fully decentralized optimization by moving beyond Local Differential Privacy (LDP) to Pairwise Network Differential Privacy (PNDP), which quantifies privacy leakage on a per-pair basis determined by graph distance. The authors introduce Muffliato, a noise-then-gossip mechanism, and provide a unified stochastic analysis for synchronous and randomized gossip across fixed and random graphs, showing that privacy amplification scales with node distance and is governed by graph topology (e.g., expander properties). They extend the framework to private decentralized GD/SGD, deriving near-central DP utility guarantees with favorable dependence on the spectral gap $\lambda_W$ and maximum degree, and demonstrate substantial privacy gains in experiments on synthetic and real networks. Overall, Muffliato achieves significant privacy amplification through decentralization while preserving scalability and competitive optimization performance, with practical implications for privacy-aware distributed learning.

Abstract

Decentralized optimization is increasingly popular in machine learning for its scalability and efficiency. Intuitively, it should also provide better privacy guarantees, as nodes only observe the messages sent by their neighbors in the network graph. But formalizing and quantifying this gain is challenging: existing results are typically limited to Local Differential Privacy (LDP) guarantees that overlook the advantages of decentralization. In this work, we introduce pairwise network differential privacy, a relaxation of LDP that captures the fact that the privacy leakage from a node $u$ to a node $v$ may depend on their relative position in the graph. We then analyze the combination of local noise injection with (simple or randomized) gossip averaging protocols on fixed and random communication graphs. We also derive a differentially private decentralized optimization algorithm that alternates between local gradient descent steps and gossip averaging. Our results show that our algorithms amplify privacy guarantees as a function of the distance between nodes in the graph, matching the privacy-utility trade-off of the trusted curator, up to factors that explicitly depend on the graph topology. Finally, we illustrate our privacy gains with experiments on synthetic and real-world datasets.

Figures (7)

• Figure 1: (a) Left: Privacy loss of Muffliato in pairwise NDP on synthetic graphs (best, worst and average in error bars over nodes at a given distance), confirming a significant privacy amplification as the distance increases. (b) Middle: Privacy loss of Muffliato from a node chosen at random on a Facebook ego graph, showing that leakage is very limited outside the node's own community. (c) Right: Privacy loss and utility of Muffliato-GD when using different Erdös-Rényi graphs after each gradient step, compared to a baseline based on a trusted aggregator.
• Figure 2: Level of the privacy loss for each node (color) with respect to a fixed node in the graph. These graphs corresponds to the graphs used in Figure \ref{['fig:synthe']}: from left to right, exponential graph, Erdos-Renyi graph, geometric random graph and grid.
• Figure 3: Privacy loss for the exponential graph with respect to the number of nodes $n$ (following powers of $2$).
• Figure 4: Privacy towards all the nodes as function of the Euclidean distance in a random geometric graph. We see a high level of correlation between the Euclidean distance and the privacy loss.
• Figure 5: Privacy loss on the 9 other Facebook Ego graphs, following the same methodology as in Figure \ref{['fig:fb']}.

Theorems & Definitions (37)

• Definition 1: Gossip matrix
• Definition 2: Spectral gap
• Definition 3: Re-scaled Chebyshev polynomials
• Definition 4: Rényi Differential Privacy
• Definition 5: Pairwise Network DP
• Theorem 1
• Theorem 2: Utility analysis
• Corollary 1
• Theorem 3: Utility analysis
• Remark 1: Accelerating Randomized Muffliato