Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Privacy-Preserving Push-Pull Method for Decentralized Optimization via State Decomposition

Huqiang Cheng, Xiaofeng Liao, Huaqing Li, You Zhao

TL;DR

The paper tackles privacy in decentralized optimization over directed graphs by proposing PPSD, a gradient-tracking method that uses state decomposition to separate a communicating gradient substate from a private substate. PPSD achieves $R$-linear convergence for strongly convex and smooth objectives and protects gradient information against honest-but-curious neighbors without incurring extra computational burden. The authors provide convergence and privacy analyses and validate results through simulations, including rendezvous and decentralized linear regression tasks. The approach generalizes several existing algorithms via parameter settings and offers a robust privacy mechanism suitable for directed networks, with potential extensions to eavesdropping scenarios.

Abstract

Distributed optimization is manifesting great potential in multiple fields, e.g., machine learning, control, and resource allocation. Existing decentralized optimization algorithms require sharing explicit state information among the agents, which raises the risk of private information leakage. To ensure privacy security, combining information security mechanisms, such as differential privacy and homomorphic encryption, with traditional decentralized optimization algorithms is a commonly used means. However, this would either sacrifice optimization accuracy or incur heavy computational burden. To overcome these shortcomings, we develop a novel privacy-preserving decentralized optimization algorithm, called PPSD, that combines gradient tracking with a state decomposition mechanism. Specifically, each agent decomposes its state associated with the gradient into two substates. One substate is used for interaction with neighboring agents, and the other substate containing private information acts only on the first substate and thus is entirely agnostic to other agents. For the strongly convex and smooth objective functions, PPSD attains a $R$-linear convergence rate. Moreover, the algorithm can preserve the agents' private information from being leaked to honest-but-curious neighbors. Simulations further confirm the results.

Privacy-Preserving Push-Pull Method for Decentralized Optimization via State Decomposition

TL;DR

The paper tackles privacy in decentralized optimization over directed graphs by proposing PPSD, a gradient-tracking method that uses state decomposition to separate a communicating gradient substate from a private substate. PPSD achieves -linear convergence for strongly convex and smooth objectives and protects gradient information against honest-but-curious neighbors without incurring extra computational burden. The authors provide convergence and privacy analyses and validate results through simulations, including rendezvous and decentralized linear regression tasks. The approach generalizes several existing algorithms via parameter settings and offers a robust privacy mechanism suitable for directed networks, with potential extensions to eavesdropping scenarios.

Abstract

Distributed optimization is manifesting great potential in multiple fields, e.g., machine learning, control, and resource allocation. Existing decentralized optimization algorithms require sharing explicit state information among the agents, which raises the risk of private information leakage. To ensure privacy security, combining information security mechanisms, such as differential privacy and homomorphic encryption, with traditional decentralized optimization algorithms is a commonly used means. However, this would either sacrifice optimization accuracy or incur heavy computational burden. To overcome these shortcomings, we develop a novel privacy-preserving decentralized optimization algorithm, called PPSD, that combines gradient tracking with a state decomposition mechanism. Specifically, each agent decomposes its state associated with the gradient into two substates. One substate is used for interaction with neighboring agents, and the other substate containing private information acts only on the first substate and thus is entirely agnostic to other agents. For the strongly convex and smooth objective functions, PPSD attains a -linear convergence rate. Moreover, the algorithm can preserve the agents' private information from being leaked to honest-but-curious neighbors. Simulations further confirm the results.
Paper Structure (17 sections, 10 theorems, 54 equations, 8 figures, 2 tables, 2 algorithms)

This paper contains 17 sections, 10 theorems, 54 equations, 8 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Fundamentals
  3. Network Topology
  4. Push-Pull Method
  5. Privacy Concern
  6. Privacy-Preserving Push-Pull Method
  7. Privacy-preserving Policy
  8. Relationship with Other Methods
  9. Convergence Performance
  10. Privacy Performance
  11. Simulation Verification
  12. Privacy Preservation in the Rendezvous Problem
  13. Decentralized Linear Regression
  14. Conclusion
  15. Proof of Lemma 4
  16. ...and 2 more sections

Key Result

Lemma 1

Let the sequence $\{ \mathbf{y}^k \} _{k\in \mathbb{N}}$ be generated by PPSD and the parameters satisfy TABLE I. Then, it holds $( \mathbf{1}_{\tilde{n}}^{\top}\otimes \mathbf{I}_d ) ( \mathbf{y}^k-\nabla \mathbf{\hat{f}}( \mathbf{x}^k ) ) =\mathbf{0}_d$ for $k\in \mathbb{N}$.

Figures (8)

  • Figure 1: State decomposition diagram.
  • Figure 2: Networks with $5$ agents (a) and $500$ agents (b).
  • Figure 3: Two different gradients of agent $1$.
  • Figure 4: The information accessible to corrupted agents $4$ and $5$ are the same under two different gradients of agent $1$ shown in Fig. 3.
  • Figure 5: Performance comparison.
  • ...and 3 more figures

Theorems & Definitions (31)

  • Definition 1
  • Definition 2
  • Definition 3
  • Remark 1
  • Lemma 1
  • proof
  • Remark 2
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • ...and 21 more