Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Optimisation with Linear Equality and Inequality Constraints using PDMM

Richard Heusdens, Guoqiang Zhang

TL;DR

This work addresses distributed optimisation of a separable convex objective over a graph under linear equality and inequality constraints. It extends the primal-dual method of multipliers (PDMM) by enforcing nonnegativity on the duals and applying Peaceman-Rachford splitting to the lifted dual, introducing a reflection operator to model data exchange without slack variables. The authors prove convergence for both synchronous and stochastic update schemes, including asynchronous updates and transmission losses, under milder conditions than previous equality-constrained PDMM results. Empirical results show that IEQ-PDMM achieves notably faster convergence than extended ADMM while maintaining low communication and computation overhead, making it well-suited for robust, distributed P2P networks.

Abstract

In this paper, we consider the problem of distributed optimisation of a separable convex cost function over a graph, where every edge and node in the graph could carry both linear equality and/or inequality constraints. We show how to modify the primal-dual method of multipliers (PDMM), originally designed for linear equality constraints, such that it can handle inequality constraints as well. The proposed algorithm does not need any slack variables, which is similar to the recent work [1] which extends the alternating direction method of multipliers (ADMM) for addressing decomposable optimisation with linear equality and inequality constraints. Using convex analysis, monotone operator theory and fixed-point theory, we show how to derive the update equations of the modified PDMM algorithm by applying Peaceman-Rachford splitting to the monotonic inclusion related to the lifted dual problem. To incorporate the inequality constraints, we impose a non-negativity constraint on the associated dual variables. This additional constraint results in the introduction of a reflection operator to model the data exchange in the network, instead of a permutation operator as derived for equality constraint PDMM. Convergence for both synchronous and stochastic update schemes of PDMM are provided. The latter includes asynchronous update schemes and update schemes with transmission losses. Experiments show that PDMM converges notably faster than extended ADMM of [1].

Distributed Optimisation with Linear Equality and Inequality Constraints using PDMM

TL;DR

This work addresses distributed optimisation of a separable convex objective over a graph under linear equality and inequality constraints. It extends the primal-dual method of multipliers (PDMM) by enforcing nonnegativity on the duals and applying Peaceman-Rachford splitting to the lifted dual, introducing a reflection operator to model data exchange without slack variables. The authors prove convergence for both synchronous and stochastic update schemes, including asynchronous updates and transmission losses, under milder conditions than previous equality-constrained PDMM results. Empirical results show that IEQ-PDMM achieves notably faster convergence than extended ADMM while maintaining low communication and computation overhead, making it well-suited for robust, distributed P2P networks.

Abstract

In this paper, we consider the problem of distributed optimisation of a separable convex cost function over a graph, where every edge and node in the graph could carry both linear equality and/or inequality constraints. We show how to modify the primal-dual method of multipliers (PDMM), originally designed for linear equality constraints, such that it can handle inequality constraints as well. The proposed algorithm does not need any slack variables, which is similar to the recent work [1] which extends the alternating direction method of multipliers (ADMM) for addressing decomposable optimisation with linear equality and inequality constraints. Using convex analysis, monotone operator theory and fixed-point theory, we show how to derive the update equations of the modified PDMM algorithm by applying Peaceman-Rachford splitting to the monotonic inclusion related to the lifted dual problem. To incorporate the inequality constraints, we impose a non-negativity constraint on the associated dual variables. This additional constraint results in the introduction of a reflection operator to model the data exchange in the network, instead of a permutation operator as derived for equality constraint PDMM. Convergence for both synchronous and stochastic update schemes of PDMM are provided. The latter includes asynchronous update schemes and update schemes with transmission losses. Experiments show that PDMM converges notably faster than extended ADMM of [1].
Paper Structure (21 sections, 3 theorems, 60 equations, 8 figures, 1 table, 2 algorithms)

This paper contains 21 sections, 3 theorems, 60 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Main contribution
  4. Organisation of the paper
  5. Background
  6. Notations and functional properties
  7. Monotone operators and operator splitting
  8. Problem Setting
  9. Operater splitting of the lifted dual function
  10. Equality and inequality constraints
  11. Node constraints
  12. Relation with ADMM
  13. Convergence of (in)equality-constraint PDMM
  14. Stochastic coordinate descent
  15. Asynchronous IEQ-PDMM
  16. ...and 6 more sections

Key Result

Lemma 1

where $[\cdot]^+$ denotes the orthogonal projection onto the non-negative orthant.

Figures (8)

  • Figure 1: Illustration of the reflection operator $R_M$.
  • Figure 2: Demonstration of a random geometric graph with 25 nodes.
  • Figure 3: Convergence results for IEQ-PDMM for the $\ell_1$ consensus problem \ref{['eq:l1']}.
  • Figure 4: Convergence results for IEQ-PDMM for the extended $\ell_1$ consensus problem \ref{['eq:l1_monotone']}.
  • Figure 5: Convergence of IEQ-PDMM for synchronous and asynchronous update schemes with different levels of transmission loss.
  • ...and 3 more figures

Theorems & Definitions (11)

  • Definition 1: Monotone operator
  • Definition 2: Nonexpansiveness
  • Definition 3: Averaged nonexpansive operator
  • Lemma 1
  • proof
  • Proposition 1
  • proof
  • Remark 1
  • Proposition 2
  • proof
  • ...and 1 more