Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Distributed Nonconvex Optimization Enabling Communication Acceleration and Privacy Protection

Zichong Ou, Jie Lu

TL;DR

A Robust Proximal Primal dual algorithm to enhance the security of information transmission, referred to as RPP, is proposed and achieves the optimal communication complexity bound for the algorithms that allow for exchanging local variables at each iteration.

Abstract

This paper addresses a distributed nonconvex optimization problem over multi-agent networks, where each agent exchanges its local information solely with its neighbors. Given that most existing distributed nonconvex optimization algorithms are susceptible to information leakage during inter agent communications, we propose a Robust Proximal Primal dual algorithm, referred to as RPP, to enhance the security of information transmission. In contrast to many existing approaches that directly transmit local variables throughout the network, we introduce carefully designed random noises to obfuscate sensitive local information. This not only preserves privacy but also demonstrates the noise robustness of our proposed algorithm. We establish a sublinear rate at which RPP converges to a stationary solution. Moreover, by incorporating Chebyshev acceleration, an accelerated variant of RPP is developed and achieves the optimal communication complexity bound for the algorithms that allow for exchanging local deci sions at each iteration. The superior convergence performance of RPP is validated through a few numerical experiments, which also indicate that, within an appropriate range, the introduced perturbations do not impede the convergence speed of RPP.

Robust Distributed Nonconvex Optimization Enabling Communication Acceleration and Privacy Protection

TL;DR

A Robust Proximal Primal dual algorithm to enhance the security of information transmission, referred to as RPP, is proposed and achieves the optimal communication complexity bound for the algorithms that allow for exchanging local variables at each iteration.

Abstract

This paper addresses a distributed nonconvex optimization problem over multi-agent networks, where each agent exchanges its local information solely with its neighbors. Given that most existing distributed nonconvex optimization algorithms are susceptible to information leakage during inter agent communications, we propose a Robust Proximal Primal dual algorithm, referred to as RPP, to enhance the security of information transmission. In contrast to many existing approaches that directly transmit local variables throughout the network, we introduce carefully designed random noises to obfuscate sensitive local information. This not only preserves privacy but also demonstrates the noise robustness of our proposed algorithm. We establish a sublinear rate at which RPP converges to a stationary solution. Moreover, by incorporating Chebyshev acceleration, an accelerated variant of RPP is developed and achieves the optimal communication complexity bound for the algorithms that allow for exchanging local deci sions at each iteration. The superior convergence performance of RPP is validated through a few numerical experiments, which also indicate that, within an appropriate range, the introduced perturbations do not impede the convergence speed of RPP.
Paper Structure (16 sections, 7 theorems, 44 equations, 2 figures, 2 algorithms)

This paper contains 16 sections, 7 theorems, 44 equations, 2 figures, 2 algorithms.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM FORMULATION
  3. ALGORITHM DEVELOPMENT
  4. Distributed Implementation
  5. Privacy Protection Mechanism
  6. Design of communication acceleration scheme
  7. CONVERGENCE ANALYSIS
  8. NUMERICAL EXPERIMENT
  9. CONCLUSIONS
  10. APPENDIX
  11. Proof of Lemma \ref{['lemma vk+1 - vk']}
  12. Proof of Lemma \ref{['lemma ALk+1 - ALk']}
  13. Proof of Lemma \ref{['lemma tildePk+1 nonincreasing']}
  14. Proof of Lemma \ref{['lemma tildePk+1 geq f*']}
  15. Proof of Theorem \ref{['theorem UMAP-Opt sublinear']}
  16. ...and 1 more sections

Key Result

Lemma 1

Suppose Assumptions assumption smooth--assumption L hold, and consider the use of primal update final, dtildek+1 original and dk+1 original. We also suppose $\mathbf{L}= \mathbf{P} \otimes \mathbf{I}_d$, $\mathbf{G}=\alpha \mathbf{I}_{Nd}-\beta \mathbf{L}$. Then, for all $k\geq 0$, where $\kappa := \frac{\lambda_1^{\mathbf{B}}}{\rho \lambda_{N-1}^{\mathbf{L}}}$, $d_1 = \bar{M}^2+\rho^2(\eta^2+\si

Figures (2)

  • Figure 1: Convergence vs. iterations.
  • Figure 2: Convergence vs. communication rounds.

Theorems & Definitions (7)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Theorem 1
  • Lemma 5
  • Proposition 1