Differentially Private Perturbed Push-Sum Protocol and Its Application in Non-Convex Optimization
Yiming Zhou, Kaiping Xue, Enhong Chen
TL;DR
This work introduces Differentially Private Perturbed Push-Sum (DPPS), a protocol-level privacy primitive for decentralized networks that injects noise based on an efficiently computed per-round sensitivity bound. To maintain practical utility, the authors design PartPSP, a downstream non-convex optimization algorithm that uses partial communication to reduce the dimensionality of shared parameters and thereby lowers the required noise, improving convergence under a fixed privacy budget. They prove that DPPS satisfies $ rac{b}{oldsymbol{\\gamma_n}}$-differential privacy and that PartPSP converges for non-convex objectives, with a convergence bound that explicitly includes the privacy-induced term $Oigl( rac{d_s S^2}{b^2 oot 2 oigl(Tigr)}igr)$. Empirically, PartPSP demonstrates superior performance over existing privacy-preserving decentralized optimization methods, especially for larger architectures, while the sensitivity-estimation mechanism provides accurate, low-overhead privacy guarantees.
Abstract
In decentralized networks, nodes cannot ensure that their shared information will be securely preserved by their neighbors, making privacy vulnerable to inference by curious nodes. Adding calibrated random noise before communication to satisfy differential privacy offers a proven defense; however, most existing methods are tailored to specific downstream tasks and lack a general, protocol-level privacy-preserving solution. To bridge this gap, we propose Differentially Private Perturbed Push-Sum (DPPS), a lightweight differential privacy protocol for decentralized communication. Since protocol-level differential privacy introduces the unique challenge of obtaining the sensitivity for each communication round, DPPS introduces a novel sensitivity estimation mechanism that requires each node to compute and broadcast only one scalar per round, enabling rigorous differential privacy guarantees. This design allows DPPS to serve as a plug-and-play, low-cost privacy-preserving solution for downstream applications built on it. To provide a concrete instantiation of DPPS and better balance the privacy-utility trade-off, we design PartPSP, a privacy-preserving decentralized algorithm for non-convex optimization that integrates a partial communication mechanism. By partitioning model parameters into local and shared components and applying DPPS only to the shared parameters, PartPSP reduces the dimensionality of consensus data, thereby lowering the magnitude of injected noise and improving optimization performance. We theoretically prove that PartPSP converges under non-convex objectives and, with partial communication, achieves better optimization performance under the same privacy budget. Experimental results validate the effectiveness of DPPS's privacy-preserving and demonstrate that PartPSP outperforms existing privacy-preserving decentralized optimization algorithms.