Distributed Nonconvex Optimization with Double Privacy Protection and Exact Convergence

TL;DR

This work tackles privacy-preserving distributed optimization for nonconvex sum-utility objectives over multi-agent networks. It introduces DPP^2, a decentralized proximal primal-dual algorithm that enforces $\epsilon$-DP for local objectives via a two-tier privacy mechanism: first-tier variable transformations to conceal private signals, and second-tier decaying Laplace noise to guarantee differential privacy. The authors prove sublinear convergence to stationary points and linear convergence to the global optimum under the Polyak-Łojasiewicz condition, while maintaining exact convergence despite privacy-preserving perturbations. Numerical experiments on distributed nonconvex classification demonstrate that DPP^2 outperforms existing DP-based methods at the same privacy level, achieving faster and exact convergence with strong privacy guarantees."

Abstract

Motivated by the pervasive lack of privacy protection in existing distributed nonconvex optimization methods, this paper proposes a decentralized proximal primal-dual algorithm enabling double protection of privacy ($\text{DPP}^2$) for minimizing nonconvex sum-utility functions over multi-agent networks, which ensures zero leakage of critical local information during inter-agent communications. We develop a two-tier privacy protection mechanism that first merges the primal and dual variables by means of a variable transformation, followed by embedding an additional random perturbation to further obfuscate the transmitted information. We theoretically establish that $\text{DPP}^2$ ensures differential privacy for local objectives while achieving exact convergence under nonconvex settings. Specifically, $\text{DPP}^2$ converges sublinearly to a stationary point and attains a linear convergence rate under the additional Polyak-Łojasiewicz (P-Ł) condition. Finally, a numerical example demonstrates the superiority of $\text{DPP}^2$ over a number of state-of-the-art algorithms, showcasing the faster, exact convergence achieved by $\text{DPP}^2$ under the same level of differential privacy.

Figures (3)

• Figure 1: Convergence performance with respect to $\bar{r}$.
• Figure 2: Convergence performance with respect to $\bar{u}$.
• Figure 3: Convergence performance with respect to $\bar{r}$.

Theorems & Definitions (15)

• Definition 1
• Definition 2
• Lemma 1
• proof
• Lemma 2
• proof
• Theorem 1
• proof
• Theorem 2
• proof