Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A distributed semismooth Newton based augmented Lagrangian method for distributed optimization

Qihao Ma, Chengjing Wang, Peipei Tang, Dunbiao Niu, Aimin Xu

Abstract

This paper proposes a novel distributed semismooth Newton based augmented Lagrangian method for solving a class of optimization problems over networks, where the global objective is defined as the sum of locally held cost functions, and communication is restricted to neighboring agents. Specifically, we employ the augmented Lagrangian method to solve an equivalently reformulated constrained version of the original problem. Each resulting subproblem is solved inexactly via a distributed semismooth Newton method. By fully leveraging the structure of the generalized Hessian, a distributed accelerated proximal gradient method is proposed to compute the Newton direction efficiently, eliminating the need to communicate with full Hessian matrices. Theoretical results are also obtained to guarantee the convergence of the proposed algorithm. Numerical experiments demonstrate the efficiency and superiority of our algorithm compared to state-of-the-art distributed algorithms.

A distributed semismooth Newton based augmented Lagrangian method for distributed optimization

Abstract

This paper proposes a novel distributed semismooth Newton based augmented Lagrangian method for solving a class of optimization problems over networks, where the global objective is defined as the sum of locally held cost functions, and communication is restricted to neighboring agents. Specifically, we employ the augmented Lagrangian method to solve an equivalently reformulated constrained version of the original problem. Each resulting subproblem is solved inexactly via a distributed semismooth Newton method. By fully leveraging the structure of the generalized Hessian, a distributed accelerated proximal gradient method is proposed to compute the Newton direction efficiently, eliminating the need to communicate with full Hessian matrices. Theoretical results are also obtained to guarantee the convergence of the proposed algorithm. Numerical experiments demonstrate the efficiency and superiority of our algorithm compared to state-of-the-art distributed algorithms.
Paper Structure (14 sections, 6 theorems, 77 equations, 4 tables, 3 algorithms)

This paper contains 14 sections, 6 theorems, 77 equations, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. preliminaries
  3. Problem statement
  4. Problem reformulation
  5. Model description
  6. DSSNAL method
  7. ALM for problem \ref{['eq:problemBx=y']}
  8. DiSSN method for problem \ref{['eq:subproblem']}
  9. Initialization for DiSSN method
  10. Restatement of ALM
  11. Numerical experiments
  12. Huber regression problem
  13. Support vector classification problem
  14. Conclusion

Key Result

Theorem 4.1

$\phi(x)$ is $\mu$-strongly convex and $L$-smooth, where $\mu=\min\limits_{1\leq i\leq m}\mu_i$, and $L=\max\limits_{1\leq i\leq m}L_i+\sigma\|B\|^2$.

Theorems & Definitions (20)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Definition 2.5
  • Definition 3.3
  • Theorem 4.1
  • proof
  • Theorem 4.2
  • proof
  • ...and 10 more