Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Mixing-Accelerated Primal-Dual Proximal Algorithm for Distributed Nonconvex Optimization

Zichong Ou, Chenyang Qiu, Dandan Wang, Jie Lu

TL;DR

This work introduces MAP-Pro, a mixing-accelerated primal-dual proximal algorithm for decentralized nonconvex optimization, enabling nodes to cooperatively minimize a sum of smooth, possibly nonconvex costs. MAP-Pro employs an augmented-Lagrangian-like primal update plus a time-varying mixing polynomial to speed up consensus across the network, with sublinear convergence to stationary points and linear convergence to the global optimum under a Polyak-Łojasiewicz condition; Chebyshev acceleration yields MAP-Pro-CA, which further reduces network-spectral dependencies and improves communication efficiency. Theoretical results quantify convergence rates and parameter conditions, while numerical experiments on distributed binary classification show MAP-Pro-CA achieving superior convergence speed and communication efficiency compared to several state-of-the-art methods. The approach offers a scalable, communication-friendly alternative for distributed nonconvex optimization in multi-agent systems and sensor networks, with practical benefits when communication rounds are costly.

Abstract

In this paper, we develop a distributed mixing-accelerated primal-dual proximal algorithm, referred to as MAP-Pro, which enables nodes in multi-agent networks to cooperatively minimize the sum of their nonconvex, smooth local cost functions in a decentralized fashion. The proposed algorithm is constructed upon minimizing a computationally inexpensive augmented-Lagrangian-like function and incorporating a time-varying mixing polynomial to expedite information fusion across the network. The convergence results derived for MAP-Pro include a sublinear rate of convergence to a stationary solution and, under the Polyak-Łojasiewics (P-Ł) condition, a linear rate of convergence to the global optimal solution. Additionally, we may embed the well-noted Chebyshev acceleration scheme in MAP-Pro, which generates a specific sequence of mixing polynomials with given degrees and enhances the convergence performance based on MAP-Pro. Finally, we illustrate the competitive convergence speed and communication efficiency of MAP-Pro via a numerical example.

A Mixing-Accelerated Primal-Dual Proximal Algorithm for Distributed Nonconvex Optimization

TL;DR

This work introduces MAP-Pro, a mixing-accelerated primal-dual proximal algorithm for decentralized nonconvex optimization, enabling nodes to cooperatively minimize a sum of smooth, possibly nonconvex costs. MAP-Pro employs an augmented-Lagrangian-like primal update plus a time-varying mixing polynomial to speed up consensus across the network, with sublinear convergence to stationary points and linear convergence to the global optimum under a Polyak-Łojasiewicz condition; Chebyshev acceleration yields MAP-Pro-CA, which further reduces network-spectral dependencies and improves communication efficiency. Theoretical results quantify convergence rates and parameter conditions, while numerical experiments on distributed binary classification show MAP-Pro-CA achieving superior convergence speed and communication efficiency compared to several state-of-the-art methods. The approach offers a scalable, communication-friendly alternative for distributed nonconvex optimization in multi-agent systems and sensor networks, with practical benefits when communication rounds are costly.

Abstract

In this paper, we develop a distributed mixing-accelerated primal-dual proximal algorithm, referred to as MAP-Pro, which enables nodes in multi-agent networks to cooperatively minimize the sum of their nonconvex, smooth local cost functions in a decentralized fashion. The proposed algorithm is constructed upon minimizing a computationally inexpensive augmented-Lagrangian-like function and incorporating a time-varying mixing polynomial to expedite information fusion across the network. The convergence results derived for MAP-Pro include a sublinear rate of convergence to a stationary solution and, under the Polyak-Łojasiewics (P-Ł) condition, a linear rate of convergence to the global optimal solution. Additionally, we may embed the well-noted Chebyshev acceleration scheme in MAP-Pro, which generates a specific sequence of mixing polynomials with given degrees and enhances the convergence performance based on MAP-Pro. Finally, we illustrate the competitive convergence speed and communication efficiency of MAP-Pro via a numerical example.
Paper Structure (11 sections, 3 theorems, 47 equations, 1 figure, 1 table, 3 algorithms)

This paper contains 11 sections, 3 theorems, 47 equations, 1 figure, 1 table, 3 algorithms.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM FORMULATION
  3. ALGORITHM DEVELOPMENT
  4. CONVERGENCE ANALYSIS
  5. Design of Mixing Acceleration
  6. NUMERICAL EXPERIMENT
  7. CONCLUSIONS
  8. APPENDIX
  9. Proof of Lemma \ref{['lemma descent sequence']}
  10. Proof of Theorem \ref{['theorem nonconvex']}
  11. Proof of Theorem \ref{['theorem pl']}

Key Result

Lemma 1

Suppose Assumptions assumption smooth--assumption polynomial hold. Let $\{\mathbf{x}^k\}$ be the sequence generated by q0--qk+1 original. Then, for any $k \geq 0$, where

Figures (1)

  • Figure 1: Convergence performance of MAP-Pro, MAP-Pro-CA, L-ADMM, Prox-GPDA-IP, xFILTER, D-GPDA, and SUDA.

Theorems & Definitions (6)

  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Theorem 2
  • proof