Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Communication-Efficient Decentralized Optimization via Double-Communication Symmetric ADMM

Jinrui Huang, Runxiong Wu, Dong Liu, Jingguo Lan, Xueqin Wang

TL;DR

The paper tackles decentralized composite optimization over networks without a central coordinator by proposing DS-ADMM, a Symmetric ADMM-based method that threads multiple communication rounds into each iteration through a novel symmetric consensus constraint. By employing proximal linearization and a two-round communication protocol, DS-ADMM achieves faster convergence and, crucially, a net reduction in total communication compared to traditional single-round schemes. The authors establish sublinear convergence in general and linear convergence under metric subregularity, with concrete PLQ-based sufficient conditions covering common models such as Lasso, logistic regression, and SVM. Empirical results on Lasso and SVM tasks demonstrate superior efficiency in both iterations and communications across multiple decentralized baselines, highlighting the method’s practical impact for scalable, privacy-preserving distributed learning.

Abstract

This paper focuses on decentralized composite optimization over networks without a central coordinator. We propose a novel decentralized Symmetric ADMM algorithm that incorporates multiple communication rounds within each iteration, derived from a new constraint formulation that enables information exchange beyond immediate neighbors. While increasing per-iteration communication, our approach significantly reduces the total number of iterations and overall communication cost. We further design optimal communication rules that minimize the number of rounds and variables transmitted per iteration. The proposed algorithms are shown to achieve linear convergence under standard assumptions. Extensive experiments on regression and classification tasks validate the theoretical results and demonstrate superior performance compared to existing decentralized optimization methods. To our knowledge, this is the first decentralized optimization framework that achieves a net reduction in total communication by leveraging fixed multi-round communication within each iteration.

Communication-Efficient Decentralized Optimization via Double-Communication Symmetric ADMM

TL;DR

The paper tackles decentralized composite optimization over networks without a central coordinator by proposing DS-ADMM, a Symmetric ADMM-based method that threads multiple communication rounds into each iteration through a novel symmetric consensus constraint. By employing proximal linearization and a two-round communication protocol, DS-ADMM achieves faster convergence and, crucially, a net reduction in total communication compared to traditional single-round schemes. The authors establish sublinear convergence in general and linear convergence under metric subregularity, with concrete PLQ-based sufficient conditions covering common models such as Lasso, logistic regression, and SVM. Empirical results on Lasso and SVM tasks demonstrate superior efficiency in both iterations and communications across multiple decentralized baselines, highlighting the method’s practical impact for scalable, privacy-preserving distributed learning.

Abstract

This paper focuses on decentralized composite optimization over networks without a central coordinator. We propose a novel decentralized Symmetric ADMM algorithm that incorporates multiple communication rounds within each iteration, derived from a new constraint formulation that enables information exchange beyond immediate neighbors. While increasing per-iteration communication, our approach significantly reduces the total number of iterations and overall communication cost. We further design optimal communication rules that minimize the number of rounds and variables transmitted per iteration. The proposed algorithms are shown to achieve linear convergence under standard assumptions. Extensive experiments on regression and classification tasks validate the theoretical results and demonstrate superior performance compared to existing decentralized optimization methods. To our knowledge, this is the first decentralized optimization framework that achieves a net reduction in total communication by leveraging fixed multi-round communication within each iteration.

Paper Structure

This paper contains 28 sections, 9 theorems, 48 equations, 4 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Decentralized Optimization.
  4. ADMM and Symmetric ADMM
  5. Multi-Communication in Decentralized Optimization
  6. Notation
  7. Preliminaries
  8. Problem Setup
  9. Graph Topology
  10. Method
  11. Reformulating Consensus Constraints
  12. Proximal Linearization
  13. Communication Strategy
  14. Algorithm
  15. Convergence Analysis
  16. ...and 13 more sections

Key Result

Proposition 1

(1) The eigenvalues of the mixing matrix satisfy $1 = \lambda_1(W) > \lambda_2(W) \geq \dots \geq \lambda_n(W) > -1$, and the spectral gap $\rho = 1 - \max\{|\lambda_2(W)|, |\lambda_n(W)|\} > 0$; (2) $\mathrm{null}(I_n - W) = \mathrm{span}\{\mathbf{1}_m\}$ .

Figures (4)

  • Figure 1: Performance on the Lasso regression task. Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
  • Figure 2: Performance on the SVM classification task. Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
  • Figure 3: Performance on the Lasso regression task. Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.
  • Figure 4: Performance on the SVM classification task. Left: suboptimality vs. iterations. Right: suboptimality vs. communication rounds.

Theorems & Definitions (17)

  • Proposition 1
  • Proposition 2
  • Remark 1
  • Proposition 3
  • Remark 2
  • Theorem 1
  • Remark 3
  • Definition 1: Metric Subregularity
  • Theorem 2
  • Definition 2
  • ...and 7 more