Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

D-ripALM: A Tuning-friendly Decentralized Relative-Type Inexact Proximal Augmented Lagrangian Method

Jiayi Zhu, Hong Wang, Ling Liang, Lei Yang

TL;DR

This work tackles decentralized consensus optimization with possibly nonsmooth convex objectives by introducing D-ripALM, a tuning-friendly double-loop proximal augmented Lagrangian method that uses a relative-type inexactness criterion to adapt inner and outer iterations. The approach enables flexible inner solvers, requires no smoothness or strong convexity, and provides rigorous convergence guarantees to primal and dual optima under standard convexity assumptions. Empirical results show that D-ripALM achieves high-precision solutions with fewer communication rounds and reduced parameter tuning compared to state-of-the-art methods such as IDEAL, PG-EXTRA, and NIDS. The framework is practical for large-scale distributed systems and offers robustness to problem conditioning and network topology, with potential extensions to time-varying or directed networks and asynchronous settings.

Abstract

This paper proposes D-ripALM, a Decentralized relative-type inexact proximal Augmented Lagrangian Method for consensus convex optimization over multi-agent networks. D-ripALM adopts a double-loop distributed optimization framework that accommodates a wide range of inner solvers, enabling efficient treatment of both smooth and nonsmooth objectives. In contrast to existing double-loop distributed augmented Lagrangian methods, D-ripALM employs a relative-type error criterion to regulate the switching between inner and outer iterations, resulting in a more practical and tuning-friendly algorithmic framework with enhanced numerical robustness. Moreover, we establish rigorous convergence guarantees for D-ripALM under general convexity assumptions, without requiring smoothness or strong convexity conditions commonly imposed in the distributed optimization literature. Numerical experiments further demonstrate the tuning-friendly nature of D-ripALM and its efficiency in attaining high-precision solutions with fewer communication rounds.

D-ripALM: A Tuning-friendly Decentralized Relative-Type Inexact Proximal Augmented Lagrangian Method

TL;DR

This work tackles decentralized consensus optimization with possibly nonsmooth convex objectives by introducing D-ripALM, a tuning-friendly double-loop proximal augmented Lagrangian method that uses a relative-type inexactness criterion to adapt inner and outer iterations. The approach enables flexible inner solvers, requires no smoothness or strong convexity, and provides rigorous convergence guarantees to primal and dual optima under standard convexity assumptions. Empirical results show that D-ripALM achieves high-precision solutions with fewer communication rounds and reduced parameter tuning compared to state-of-the-art methods such as IDEAL, PG-EXTRA, and NIDS. The framework is practical for large-scale distributed systems and offers robustness to problem conditioning and network topology, with potential extensions to time-varying or directed networks and asynchronous settings.

Abstract

This paper proposes D-ripALM, a Decentralized relative-type inexact proximal Augmented Lagrangian Method for consensus convex optimization over multi-agent networks. D-ripALM adopts a double-loop distributed optimization framework that accommodates a wide range of inner solvers, enabling efficient treatment of both smooth and nonsmooth objectives. In contrast to existing double-loop distributed augmented Lagrangian methods, D-ripALM employs a relative-type error criterion to regulate the switching between inner and outer iterations, resulting in a more practical and tuning-friendly algorithmic framework with enhanced numerical robustness. Moreover, we establish rigorous convergence guarantees for D-ripALM under general convexity assumptions, without requiring smoothness or strong convexity conditions commonly imposed in the distributed optimization literature. Numerical experiments further demonstrate the tuning-friendly nature of D-ripALM and its efficiency in attaining high-precision solutions with fewer communication rounds.
Paper Structure (11 sections, 2 theorems, 27 equations, 2 tables, 1 algorithm)

This paper contains 11 sections, 2 theorems, 27 equations, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation.
  3. Problem setup
  4. Assumptions on the objective functions and network
  5. Consensus reformulation
  6. A decentralized relative-type inexact proximal ALM
  7. Convergence analysis
  8. Numerical experiments
  9. Comparison with IDEAL
  10. Comparison with PG-EXTRA and NIDS
  11. Conclusions

Key Result

Theorem 4.1

Let $\rho\in[0,1)$, $\{\sigma_{k}\}$ be a positive sequence satisfying $\sigma_k\geq\sigma_{\min}>0$ for all $k\geq0$, and $\{\tau_k\}$ be a positive sequence satisfying Let $\{\mathbf{x}^{k}\}$, $\{\Delta^{k}\}$, $\{\mathbf{w}^{k}\}$, and $\{\mathbf{y}^{k}\}$ be the iterates generated by D-ripALM in Algorithm alg:D-ripALM. If $\ell$ admits a saddle point, i.e., $(\partial\ell)^{-1}(\bm{0},\bm{0}

Theorems & Definitions (2)

  • Theorem 4.1
  • Theorem 4.2: Asymptotic superlinear convergence