Low-Cost Privacy-Preserving Decentralized Learning
Sayan Biswas, Davide Frey, Romaric Gaudel, Anne-Marie Kermarrec, Dimitri Lerévérend, Rafael Pires, Rishi Sharma, François Taïani
TL;DR
Zip-DL tackles privacy in decentralized learning by enabling a single-round gossip averaging pass ($s=1$) with locally correlated zero-sum noise, yielding formal PNDP guarantees without coordination. The method preserves convergence by ensuring noise cancels in the global average, while concentrating the privacy burden in pairwise, neighborhood-level disclosures. The authors provide convergence bounds under standard smoothness/convexity assumptions and PNDP-based privacy analysis, and demonstrate through CIFAR-10 and MovieLens that Zip-DL achieves a superior privacy-utility trade-off compared with state-of-the-art baselines while incurring low communication costs. Overall, Zip-DL offers a practical, efficient approach to privacy-preserving decentralized learning in real-world networks, enabling strong protection with minimal overhead.
Abstract
Decentralized learning (DL) is an emerging paradigm of collaborative machine learning that enables nodes in a network to train models collectively without sharing their raw data or relying on a central server. This paper introduces Zip-DL, a privacy-aware DL algorithm that leverages correlated noise to achieve robust privacy against local adversaries while ensuring efficient convergence at low communication costs. By progressively neutralizing the noise added during distributed averaging, Zip-DL combines strong privacy guarantees with high model accuracy. Its design requires only one communication round per gradient descent iteration, significantly reducing communication overhead compared to competitors. We establish theoretical bounds on both convergence speed and privacy guarantees. Moreover, extensive experiments demonstrating Zip-DL's practical applicability make it outperform state-of-the-art methods in the accuracy vs. vulnerability trade-off. Specifically, Zip-DL (i) reduces membership-inference attack success rates by up to 35% compared to baseline DL, (ii) decreases attack efficacy by up to 13% compared to competitors offering similar utility, and (iii) achieves up to 59% higher accuracy to completely nullify a basic attack scenario, compared to a state-of-the-art privacy-preserving approach under the same threat model. These results position Zip-DL as a practical and efficient solution for privacy-preserving decentralized learning in real-world applications.