Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Optimization over Lossy Networks via Relaxed Peaceman-Rachford Splitting: a Robust ADMM Approach

Nicola Bastianello, Marco Todescato, Ruggero Carli, Luca Schenato

Abstract

In this work we address the problem of distributed optimization of the sum of convex cost functions in the context of multi-agent systems over lossy communication networks. Building upon operator theory, first, we derive an ADMM-like algorithm that we refer to as relaxed ADMM (R-ADMM) via a generalized Peaceman-Rachford Splitting operator on the Lagrange dual formulation of the original optimization problem. This specific algorithm depends on two parameters, namely the averaging coefficient $α$ and the augmented Lagrangian coefficient $ρ$. We show that by setting $α=1/2$ we recover the standard ADMM algorithm as a special case of our algorithm. Moreover, by properly manipulating the proposed R-ADMM, we are able to provide two alternative ADMM-like algorithms that present easier implementation and reduced complexity in terms of memory, communication and computational requirements. Most importantly the latter of these two algorithms provides the first ADMM-like algorithm which has guaranteed convergence even in the presence of lossy communication under the same assumption of standard ADMM with lossless communication. Finally, this work is complemented with a set of compelling numerical simulations of the proposed algorithms over cycle graphs and random geometric graphs subject to i.i.d. random packet losses.

Distributed Optimization over Lossy Networks via Relaxed Peaceman-Rachford Splitting: a Robust ADMM Approach

Abstract

In this work we address the problem of distributed optimization of the sum of convex cost functions in the context of multi-agent systems over lossy communication networks. Building upon operator theory, first, we derive an ADMM-like algorithm that we refer to as relaxed ADMM (R-ADMM) via a generalized Peaceman-Rachford Splitting operator on the Lagrange dual formulation of the original optimization problem. This specific algorithm depends on two parameters, namely the averaging coefficient and the augmented Lagrangian coefficient . We show that by setting we recover the standard ADMM algorithm as a special case of our algorithm. Moreover, by properly manipulating the proposed R-ADMM, we are able to provide two alternative ADMM-like algorithms that present easier implementation and reduced complexity in terms of memory, communication and computational requirements. Most importantly the latter of these two algorithms provides the first ADMM-like algorithm which has guaranteed convergence even in the presence of lossy communication under the same assumption of standard ADMM with lossless communication. Finally, this work is complemented with a set of compelling numerical simulations of the proposed algorithms over cycle graphs and random geometric graphs subject to i.i.d. random packet losses.

Paper Structure

This paper contains 23 sections, 5 theorems, 63 equations, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background on Splitting Operators
  3. Definitions and Properties
  4. Finding the Fixed Points of Nonexpansive Operators
  5. From the ADMM to the Relaxed ADMM
  6. The ADMM Algorithm
  7. The Relaxed ADMM
  8. Distributed Consensus Optimization
  9. Problem Formulation
  10. R-ADMM for Convex Distributed Optimization
  11. Distributed R-ADMM over lossy networks
  12. Simulations
  13. Conclusions and Future Directions
  14. Derivation of Algorithm \ref{['alg:r-admm-three-eqs']}
  15. Equation \ref{['eq:r-admm-1']}
  16. ...and 8 more sections

Key Result

Proposition 1

The implementation of the R-ADMM algorithm described in the set of five equations given in eq:psi-update, eq:xi-update and eq:prs-3 applied to the dual of problem eq:primal-indicator-f, reduces to alternating between the following two updates for all $i \in V$, and for all $(i,j) \in \mathcal{E}$.$\square$

Figures (4)

  • Figure 1: Relationships between the algorithms.
  • Figure 1: Evolution, in log-scale, of the relative error of Alg. \ref{['alg:robust-smart-distributed-r-admm']} computed w.r.t. the unique optimal solution $x^*$ as function of different values of packet loss probability $p$ for step size $\alpha=1$ and penalty $\rho=1$. Average over 100 Monte Carlo runs.
  • Figure 2: Stability boundaries of Alg. \ref{['alg:robust-smart-distributed-r-admm']} as function of the step size $\alpha$ and the penalty $\rho$ for different values of loss probability $p$ for the family of random geometric graphs. Average over 100 Monte Carlo runs.
  • Figure 3: Evolution, in log-scale, of the relative error of Alg. \ref{['alg:robust-smart-distributed-r-admm']} computed w.r.t. the unique optimal solution $x^*$ as function of different values of the step size $\alpha$, with fixed packet loss probability $p=0.6$ and penalty $\rho=1$. Average over 100 Monte Carlo runs.

Theorems & Definitions (13)

  • Definition 1: Nonexpansive operators
  • Definition 2: $\alpha$-averaged operators
  • Definition 3: Proximal and reflective operators
  • Remark 1
  • Proposition 1
  • Remark 2
  • Proposition 2
  • Proposition 3
  • Remark 3
  • Remark 4
  • ...and 3 more