Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence Theory of Flexible ALADIN for Distributed Optimization

Xu Du, Xiaohua Zhou, Shijie Zhu

TL;DR

Flexible ALADIN is proposed, a random polling variant of ALADIN, and a rigorous convergence analysis is presented, including global convergence for convex problems and local convergence for non-convex problems.

Abstract

The Augmented Lagrangian Alternating Direction Inexact Newton (ALADIN) method is a cutting-edge distributed optimization algorithm known for its superior numerical performance. It relies on each agent transmitting information to a central coordinator for data exchange. However, in practical network optimization and federated learning, unreliable information transmission often leads to packet loss, posing challenges for the convergence analysis of ALADIN. To address this issue, this paper proposes Flexible ALADIN, a random polling variant of ALADIN, and presents a rigorous convergence analysis, including global convergence for convex problems and local convergence for non-convex problems.

Convergence Theory of Flexible ALADIN for Distributed Optimization

TL;DR

Flexible ALADIN is proposed, a random polling variant of ALADIN, and a rigorous convergence analysis is presented, including global convergence for convex problems and local convergence for non-convex problems.

Abstract

The Augmented Lagrangian Alternating Direction Inexact Newton (ALADIN) method is a cutting-edge distributed optimization algorithm known for its superior numerical performance. It relies on each agent transmitting information to a central coordinator for data exchange. However, in practical network optimization and federated learning, unreliable information transmission often leads to packet loss, posing challenges for the convergence analysis of ALADIN. To address this issue, this paper proposes Flexible ALADIN, a random polling variant of ALADIN, and presents a rigorous convergence analysis, including global convergence for convex problems and local convergence for non-convex problems.

Paper Structure

This paper contains 12 sections, 3 theorems, 35 equations, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. Flexible ALADIN
  5. Convergence Theory of FC-ALADIN
  6. Global Convergence of Exact FC-ALADIN for Strongly Convex Cases
  7. Local Convergence of Exact FC-ALADIN for Smooth Non-convex Cases
  8. Convergence of Inexact FC-ALADIN for Strictly Convex Cases
  9. Conclusion
  10. Proof of Theorem \ref{['theorem: lya']}
  11. Proof of Theorem \ref{['theorem: smooth2']}
  12. Proof of Theorem \ref{['theorem: theorem3']}

Key Result

Theorem 1

Let the local objectives $f_i$s in Problem eq: DOPT_C be closed, proper, smooth, and strongly convex. Let $B_i\in \mathbb{S}^n_{++}$ be proper, symmetric, and strictly positive definite constant matrices. Define $x_i^*=y^*$ and $\lambda_i^*$ as the optimal primal and dual solutions of eq: DOPT_C. Gi where $\alpha=\left( \frac{p}{1+\delta} + (1-p) \right)<1$.

Theorems & Definitions (3)

  • Theorem 1
  • Theorem 2
  • Theorem 3