Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentially Private Decentralized Optimization with Relay Communication

Luqing Wang, Luyao Guo, Shaofu Yang, Xinli Shi

TL;DR

This work tackles privacy risks in decentralized optimization by introducing Privacy Leakage Frequency ($PLF$) and a privacy-preserving relay-based algorithm, DP-RECAL. DP-RECAL combines a relay communication mechanism with an AFBS-inspired primal–dual proximal framework and adds Gaussian noise to relayed messages, achieving lower $PLF$ and improved privacy budgets relative to existing methods. The authors prove global convergence for general convex problems with uncoordinated stepsizes and establish a linear rate under metric subregularity, complemented by a $\rho$-zCDP privacy analysis that ties the overall $(\epsilon,\delta)$-DP guarantee to $PLF$. Numerical experiments on least-squares and real datasets show that DP-RECAL delivers competitive accuracy with reduced privacy leakage and demonstrates resilience to gradient-leakage attacks, highlighting practical impact for secure, scalable decentralized optimization.

Abstract

Security concerns in large-scale networked environments are becoming increasingly critical. To further improve the algorithm security from the design perspective of decentralized optimization algorithms, we introduce a new measure: Privacy Leakage Frequency (PLF), which reveals the relationship between communication and privacy leakage of algorithms, showing that lower PLF corresponds to lower privacy budgets. Based on such assertion, a novel differentially private decentralized primal--dual algorithm named DP-RECAL is proposed to take advantage of operator splitting method and relay communication mechanism to experience less PLF so as to reduce the overall privacy budget. To the best of our knowledge, compared with existing differentially private algorithms, DP-RECAL presents superior privacy performance and communication complexity. In addition, with uncoordinated network-independent stepsizes, we prove the convergence of DP-RECAL for general convex problems and establish a linear convergence rate under the metric subregularity. Evaluation analysis on least squares problem and numerical experiments on real-world datasets verify our theoretical results and demonstrate that DP-RECAL can defend some classical gradient leakage attacks.

Differentially Private Decentralized Optimization with Relay Communication

TL;DR

This work tackles privacy risks in decentralized optimization by introducing Privacy Leakage Frequency () and a privacy-preserving relay-based algorithm, DP-RECAL. DP-RECAL combines a relay communication mechanism with an AFBS-inspired primal–dual proximal framework and adds Gaussian noise to relayed messages, achieving lower and improved privacy budgets relative to existing methods. The authors prove global convergence for general convex problems with uncoordinated stepsizes and establish a linear rate under metric subregularity, complemented by a -zCDP privacy analysis that ties the overall -DP guarantee to . Numerical experiments on least-squares and real datasets show that DP-RECAL delivers competitive accuracy with reduced privacy leakage and demonstrates resilience to gradient-leakage attacks, highlighting practical impact for secure, scalable decentralized optimization.

Abstract

Security concerns in large-scale networked environments are becoming increasingly critical. To further improve the algorithm security from the design perspective of decentralized optimization algorithms, we introduce a new measure: Privacy Leakage Frequency (PLF), which reveals the relationship between communication and privacy leakage of algorithms, showing that lower PLF corresponds to lower privacy budgets. Based on such assertion, a novel differentially private decentralized primal--dual algorithm named DP-RECAL is proposed to take advantage of operator splitting method and relay communication mechanism to experience less PLF so as to reduce the overall privacy budget. To the best of our knowledge, compared with existing differentially private algorithms, DP-RECAL presents superior privacy performance and communication complexity. In addition, with uncoordinated network-independent stepsizes, we prove the convergence of DP-RECAL for general convex problems and establish a linear convergence rate under the metric subregularity. Evaluation analysis on least squares problem and numerical experiments on real-world datasets verify our theoretical results and demonstrate that DP-RECAL can defend some classical gradient leakage attacks.
Paper Structure (31 sections, 13 theorems, 101 equations, 9 figures, 5 tables, 1 algorithm)

This paper contains 31 sections, 13 theorems, 101 equations, 9 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem formulation
  3. Optimization model
  4. Threat model
  5. Differential privacy
  6. Proposal of PLF
  7. DP-RECAL
  8. Algorithm construction
  9. Decentralized implementation via relay communication
  10. Discussion
  11. Differential privacy
  12. Convergence analysis
  13. Concise formulation of DP-RECAL
  14. Global convergence
  15. Linear convergence rate
  16. ...and 16 more sections

Key Result

Proposition 1

Dwork2014 If randomized mechanisms $\mathcal{M}_1, \mathcal{M}_2, \cdots, \mathcal{M}_{K}$ are respectively $(\epsilon_1,\delta)$-DP, $(\epsilon_2,\delta)$-DP, $\cdots$, $(\epsilon_{\mathrm{K}},\delta)$-DP, the algorithm satisfies $(\sum_{k=1}^{\mathrm{K}}\epsilon_k,\delta)$-$DP$.

Figures (9)

  • Figure 1: An example of DP-RECAL implementation with external adversaries
  • Figure 2: Convergence curves of DP-RECAL with various privacy budgets for the linear regression problem on four types of datasets.
  • Figure 3: Convergence performance and classification accuracy of various differentially private algorithms with privacy budget $\epsilon = 12$ for the linear regression problems on four types of datasets.
  • Figure 4: Comparisons of the results achieved by the adversary with DLG method Zhu2019 on classification of three datasets performed by DP-RECAL and RECAL.
  • Figure 5: Convergence curves of DP-RECAL with various privacy budgets for the logistic regression problem on four types of datasets.
  • ...and 4 more figures

Theorems & Definitions (20)

  • Definition 1
  • Proposition 1: Sequential combinations
  • Proposition 2: Parallel combinations
  • Theorem 1
  • Remark 1
  • Definition 2
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • ...and 10 more