Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Decentralized Constraint-Coupled Optimization: A Dual$^2$ Approach

Jingwang Li, Vincent Lau

TL;DR

<3-5 sentence high-level summary> The paper addresses decentralized optimization with coupled constraints by introducing a novel dual^2 framework that splits the problem into a smooth unconstrained part and a saddle-point problem, enabling acceleration via Nesterov methods. It proposes two algorithms, iD2A and MiD2A, proving asymptotic convergence under mild convexity on h and linear convergence under specific structural cases, with detailed outer/inner complexities and subproblem solver choices. Through theoretical analysis and experiments on decentralized elastic net and constrained regression, the work demonstrates substantial improvements in communication and computation costs over state-of-the-art methods. The results offer practical guidance on rho selection and subproblem solver choice, and suggest avenues for future work on nonconvex settings and directed networks.

Abstract

In this paper, we focus on a class of decentralized constraint-coupled optimization problem: $\min_{x_i \in \mathbb{R}^{d_i}, i \in \mathcal{I}; y \in \mathbb{R}^p}$ $\sum_{i=1}^n\left(f_i(x_i) + g_i(x_i)\right) + h(y) \ \text{s.t.} \ \sum_{i=1}^{n}A_ix_i = y$, over an undirected and connected network of $n$ agents. Here, $f_i$, $g_i$, and $A_i$ represent private information of agent $i \in \mathcal{I} = \{1, \cdots, n\}$, while $h$ is public for all agents. Building on a novel dual$^2$ approach, we develop two accelerated algorithms to solve this problem: the inexact Dual$^2$ Accelerated (iD2A) gradient method and the Multi-consensus inexact Dual$^2$ Accelerated (MiD2A) gradient method. We demonstrate that both iD2A and MiD2A can guarantee asymptotic convergence under a milder condition on $h$ compared to existing algorithms. Furthermore, under additional assumptions, we establish linear convergence rates and derive significantly lower communication and computational complexity bounds than those of existing algorithms. Several numerical experiments validate our theoretical analysis and demonstrate the practical superiority of the proposed algorithms.

Accelerated Decentralized Constraint-Coupled Optimization: A Dual$^2$ Approach

TL;DR

<3-5 sentence high-level summary> The paper addresses decentralized optimization with coupled constraints by introducing a novel dual^2 framework that splits the problem into a smooth unconstrained part and a saddle-point problem, enabling acceleration via Nesterov methods. It proposes two algorithms, iD2A and MiD2A, proving asymptotic convergence under mild convexity on h and linear convergence under specific structural cases, with detailed outer/inner complexities and subproblem solver choices. Through theoretical analysis and experiments on decentralized elastic net and constrained regression, the work demonstrates substantial improvements in communication and computation costs over state-of-the-art methods. The results offer practical guidance on rho selection and subproblem solver choice, and suggest avenues for future work on nonconvex settings and directed networks.

Abstract

In this paper, we focus on a class of decentralized constraint-coupled optimization problem: , over an undirected and connected network of agents. Here, , , and represent private information of agent , while is public for all agents. Building on a novel dual approach, we develop two accelerated algorithms to solve this problem: the inexact Dual Accelerated (iD2A) gradient method and the Multi-consensus inexact Dual Accelerated (MiD2A) gradient method. We demonstrate that both iD2A and MiD2A can guarantee asymptotic convergence under a milder condition on compared to existing algorithms. Furthermore, under additional assumptions, we establish linear convergence rates and derive significantly lower communication and computational complexity bounds than those of existing algorithms. Several numerical experiments validate our theoretical analysis and demonstrate the practical superiority of the proposed algorithms.
Paper Structure (47 sections, 21 theorems, 109 equations, 3 figures, 7 tables, 7 algorithms)

This paper contains 47 sections, 21 theorems, 109 equations, 3 figures, 7 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Our Contributions
  4. Preliminaries
  5. Notations
  6. Convex Analysis
  7. Algorithm Development
  8. Basic Assumptions
  9. A Dual$^2$ Approach to Achieve Acceleration
  10. Inexact Dual$^2$ Accelerated Gradient Method
  11. Multi-Consensus Inexact Dual$^2$ Accelerated Gradient Method
  12. Convergence Analysis: Convergence Rates
  13. Convergence under the General Convex Case
  14. Convergence Rates Under the Strongly Convex Case
  15. Convergence Analysis: Overall Complexities
  16. ...and 32 more sections

Key Result

Lemma 1

vandenberghe2022om Assume that $f: \mathbb{R}^n \rightarrow \mathbb{R} \cup \{+\infty\}$ is closed proper and $\mu$-strongly convex with $\mu>0$, then (1) $\text{dom} \, f^* = \mathbb{R}^n$; (2) $f^*$ is differentiable on $\mathbb{R}^n$ with $\nabla f^*(y) = \arg\max_{x \in \text{dom} \, f} y^{\top}

Figures (3)

  • Figure 1: Results of Experiment I: Decentralized Elastic Net Regression ($n = 8, p = 20, d = 9, \kappa_C = 25, \kappa_f = 1, \kappa_{pd} = 98603$).
  • Figure 2: Results of Experiment II: Decentralized Constrained Linear Regression ($n = 8, p = 9, d = 9, \kappa_C = 25, \kappa_f = 1, \kappa_{\mathbf{A}_{\rho}} = 8.24 \times 10^{15}, \kappa_{\mathbf{A}'_{\rho}} = 4.89 \times 10^{16}$).
  • Figure 3: Results of Experiment III: Decentralized Resource Allocation ($n = 20, p = 10, d = 40, \kappa_C = 99, \kappa_f = 1000, \kappa_{A} = 8)$.

Theorems & Definitions (49)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Remark 1
  • Remark 2
  • Lemma 5
  • proof
  • Lemma 6
  • proof
  • ...and 39 more