Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Implicit Tracking-Based Distributed Constraint-Coupled Optimization

Jingwang Li, Housheng Su

TL;DR

The novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the implicit tracking-based distributed augmented primal-dual gradient dynamics (IDEA), the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions.

Abstract

A class of distributed optimization problem with a globally coupled equality constraint and local constrained sets is studied in this paper. For its special case where local constrained sets are absent, an augmented primal-dual gradient dynamics is proposed and analyzed, but it cannot be implemented distributedly since the violation of the coupled constraint needs to be used. Benefiting from the brand-new comprehending of a classical distributed unconstrained optimization algorithm, the novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the \underline{i}mplicit tracking-based \underline{d}istribut\underline{e}d \underline{a}ugmented primal-dual gradient dynamics (IDEA). A projected variant of IDEA, i.e., Proj-IDEA, is further designed to deal with the general case where local constrained sets exist. With the aid of the Lyapunov stability theory, the convergences of IDEA and Pro-IDEA over undigraphs and digraphs are analyzed respectively. As far as we know, Proj-IDEA is the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions. Besides, if local cost functions are strongly convex and smooth, IDEA can achieve exponential convergence with a weaker condition about the coupled constraint. Finally, numerical experiments are taken to corroborate our theoretical results.

Implicit Tracking-Based Distributed Constraint-Coupled Optimization

TL;DR

The novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the implicit tracking-based distributed augmented primal-dual gradient dynamics (IDEA), the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions.

Abstract

A class of distributed optimization problem with a globally coupled equality constraint and local constrained sets is studied in this paper. For its special case where local constrained sets are absent, an augmented primal-dual gradient dynamics is proposed and analyzed, but it cannot be implemented distributedly since the violation of the coupled constraint needs to be used. Benefiting from the brand-new comprehending of a classical distributed unconstrained optimization algorithm, the novel implicit tracking approach is proposed to track the violation distributedly, which leads to the birth of the \underline{i}mplicit tracking-based \underline{d}istribut\underline{e}d \underline{a}ugmented primal-dual gradient dynamics (IDEA). A projected variant of IDEA, i.e., Proj-IDEA, is further designed to deal with the general case where local constrained sets exist. With the aid of the Lyapunov stability theory, the convergences of IDEA and Pro-IDEA over undigraphs and digraphs are analyzed respectively. As far as we know, Proj-IDEA is the first constant step-size distributed algorithm which can solve the studied problem without the need of the strict convexity of local cost functions. Besides, if local cost functions are strongly convex and smooth, IDEA can achieve exponential convergence with a weaker condition about the coupled constraint. Finally, numerical experiments are taken to corroborate our theoretical results.
Paper Structure (14 sections, 16 theorems, 156 equations, 4 figures)

This paper contains 14 sections, 16 theorems, 156 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations
  4. Graph Theory
  5. Convex Analysis
  6. Algorithm Design
  7. Centralized Augmented Primal-dual Gradient Dynamics
  8. Implicit Tracking Approach
  9. Implicit Tracking-based Distributed Augmented Primal-dual Gradient Dynamics
  10. Convergence Analysis
  11. Convergence of IDEA
  12. Convergence of Proj-IDEA
  13. Numerical Experiments
  14. Conclusion

Key Result

Lemma 1

nesterov2018lectures Given a convex set $\mathcal{X} \subseteq \mathbb{R}^{d}$ and a differentiable function $f:\mathbb{R}^d \rightarrow \mathbb{R}$, if $f$ is $l$-smooth on $\mathcal{X}$ with $l>0$, then If $f$ is convex on $\mathcal{X}$, then If $f$ is $\mu$-strongly convex on $\mathcal{X}$ with $\mu>0$, then

Figures (4)

  • Figure 1: Experiment results of Case 1. (a) Circle graph, $\eta_2(L) \approx 0.02$. (b) Random graph with $p=0.05$, $\eta_2(L) \approx 0.21$. (c) Random graph with $p=0.1$, $\eta_2(L) \approx 0.54$. (d) Random graph with $p=0.3$, $\eta_2(L) \approx 6.34$.
  • Figure 2: Experiment results of Case 2. (a) Circle graph, $\eta_2(L) \approx 0.02$. (b) Random graph with $p=0.05$, $\eta_2(L) \approx 0.21$. (c) Random graph with $p=0.1$, $\eta_2(L) \approx 0.54$. (d) Random graph with $p=0.3$, $\eta_2(L) \approx 6.34$.
  • Figure 3: Experiment results of Case 3. (a) Directed circle graph. (b) Directed exponential graph with $e=2$. (c) Directed exponential graph with $e=4$. (d) Directed exponential graph with $e=6$.
  • Figure 4: Experiment results of Case 4. (a) Directed circle graph. (b) Directed exponential graph with $e=2$. (c) Directed exponential graph with $e=4$. (d) Directed exponential graph with $e=6$.

Theorems & Definitions (26)

  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Proposition 1
  • Proposition 2
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • ...and 16 more