Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

dHPR: A Distributed Halpern Peaceman--Rachford Method for Non-smooth Distributed Optimization Problems

Zhangcheng Feng, Defeng Sun, Yancheng Yuan, Guojun Zhang

TL;DR

The paper addresses distributed convex composite optimization over networks with non-smooth objectives by introducing dHPR, a distributed Halpern–Peaceman–Rachford method. By combining Halpern iteration with a symmetric Gauss–Seidel–based decoupling, dHPR achieves non-ergodic $O(1/k)$ convergence in both the KKT residual and the dual objective error, while enabling fully decentralized, proximal-term–free per-agent updates. The authors provide convergence proofs, an efficient decentralized implementation, and extensive experiments on distributed LASSO, group LASSO, and $L_1$-regularized logistic regression, demonstrating superior performance over state-of-the-art methods such as NIDS and PG-EXTRA. The work advances scalable distributed optimization by delivering fast convergence without large proximal terms and offers practical impact for decentralized machine learning and sensor networks.

Abstract

This paper introduces the distributed Halpern Peaceman--Rachford (dHPR) method, an efficient algorithm for solving distributed convex composite optimization problems with non-smooth objectives, which achieves a non-ergodic $O(1/k)$ iteration complexity regarding Karush--Kuhn--Tucker residual. By leveraging the symmetric Gauss--Seidel decomposition, the dHPR effectively decouples the linear operators in the objective functions and consensus constraints while maintaining parallelizability and avoiding additional large proximal terms, leading to a decentralized implementation with provably fast convergence. The superior performance of dHPR is demonstrated through comprehensive numerical experiments on distributed LASSO, group LASSO, and $L_1$-regularized logistic regression problems.

dHPR: A Distributed Halpern Peaceman--Rachford Method for Non-smooth Distributed Optimization Problems

TL;DR

The paper addresses distributed convex composite optimization over networks with non-smooth objectives by introducing dHPR, a distributed Halpern–Peaceman–Rachford method. By combining Halpern iteration with a symmetric Gauss–Seidel–based decoupling, dHPR achieves non-ergodic convergence in both the KKT residual and the dual objective error, while enabling fully decentralized, proximal-term–free per-agent updates. The authors provide convergence proofs, an efficient decentralized implementation, and extensive experiments on distributed LASSO, group LASSO, and -regularized logistic regression, demonstrating superior performance over state-of-the-art methods such as NIDS and PG-EXTRA. The work advances scalable distributed optimization by delivering fast convergence without large proximal terms and offers practical impact for decentralized machine learning and sensor networks.

Abstract

This paper introduces the distributed Halpern Peaceman--Rachford (dHPR) method, an efficient algorithm for solving distributed convex composite optimization problems with non-smooth objectives, which achieves a non-ergodic iteration complexity regarding Karush--Kuhn--Tucker residual. By leveraging the symmetric Gauss--Seidel decomposition, the dHPR effectively decouples the linear operators in the objective functions and consensus constraints while maintaining parallelizability and avoiding additional large proximal terms, leading to a decentralized implementation with provably fast convergence. The superior performance of dHPR is demonstrated through comprehensive numerical experiments on distributed LASSO, group LASSO, and -regularized logistic regression problems.

Paper Structure

This paper contains 21 sections, 4 theorems, 33 equations, 6 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. The Communication Graph
  4. Reformulations of DOPs and dual DOPs
  5. dHPR: A distributed HPR method
  6. dHPR
  7. Convergence Analysis
  8. An Efficient Implementation of dHPR
  9. Numerical Experiment
  10. Experimental Setup
  11. Communication graph
  12. Initial values, metric & termination criteria
  13. Baseline algorithms
  14. The restart and $\sigma$ update strategies
  15. Distributed LASSO and Group LASSO
  16. ...and 6 more sections

Key Result

Proposition 1

Let $\mathcal{T}=\mathcal{S}+\hat{\mathcal{S}}$ with $\mathcal{S}$ and $\hat{\mathcal{S}}$ given in eq:S&Shat. Then for any $k\geq0$, the update of $(\bar{\boldsymbol z}^{k+1},\, \bar{\boldsymbol w}^{k+1})$ in Algorithm alg:sphpr, i.e., is equivalent to the following updates: Moreover, $\mathcal{T}+\sigma\boldsymbol A_U^\top\boldsymbol A_U\succ\boldsymbol0$, where $\boldsymbol A_U:=\in\mathbb R^

Figures (6)

  • Figure 1: Distributed LASSO
  • Figure 2: Distributed group LASSO
  • Figure 3: Distributed $L_1$-regularized logistic regression
  • Figure 4: Effect of topologies
  • Figure 5: Comparison of dHPR and dual L-HPR
  • ...and 1 more figures

Theorems & Definitions (10)

  • Proposition 1: li2019block
  • Remark 1: sGS decomposition
  • Remark 2: Connection with existing algorithms
  • Theorem 1
  • Theorem 2
  • Remark 3
  • Lemma 1
  • proof
  • proof
  • proof