Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Conjugate Gradient Method via Conjugate Direction Tracking

Ola Shorinwa, Mac Schwager

TL;DR

The paper tackles distributed optimization over a connected network where each agent holds only local data and no central coordinator is available. It introduces DC-Grad, a distributed conjugate gradient method that tracks the network-average conjugate direction via dynamic average consensus and allows uncoordinated constant step-sizes. The authors establish convergence to the aggregate optimum, prove consensus of local iterates, and demonstrate favorable performance in distributed state estimation and robust least-squares simulations, particularly on densely connected graphs where it reduces communication overhead. These results provide a privacy-preserving, faster-than-basic-first-order alternative for distributed optimization in nonlinear convex settings.

Abstract

We present a distributed conjugate gradient method for distributed optimization problems, where each agent computes an optimal solution of the problem locally without any central computation or coordination, while communicating with its immediate, one-hop neighbors over a communication network. Each agent updates its local problem variable using an estimate of the average conjugate direction across the network, computed via a dynamic consensus approach. Our algorithm enables the agents to use uncoordinated step-sizes. We prove convergence of the local variable of each agent to the optimal solution of the aggregate optimization problem, without requiring decreasing step-sizes. In addition, we demonstrate the efficacy of our algorithm in distributed state estimation problems, and its robust counterparts, where we show its performance compared to existing distributed first-order optimization methods.

Distributed Conjugate Gradient Method via Conjugate Direction Tracking

TL;DR

The paper tackles distributed optimization over a connected network where each agent holds only local data and no central coordinator is available. It introduces DC-Grad, a distributed conjugate gradient method that tracks the network-average conjugate direction via dynamic average consensus and allows uncoordinated constant step-sizes. The authors establish convergence to the aggregate optimum, prove consensus of local iterates, and demonstrate favorable performance in distributed state estimation and robust least-squares simulations, particularly on densely connected graphs where it reduces communication overhead. These results provide a privacy-preserving, faster-than-basic-first-order alternative for distributed optimization in nonlinear convex settings.

Abstract

We present a distributed conjugate gradient method for distributed optimization problems, where each agent computes an optimal solution of the problem locally without any central computation or coordination, while communicating with its immediate, one-hop neighbors over a communication network. Each agent updates its local problem variable using an estimate of the average conjugate direction across the network, computed via a dynamic consensus approach. Our algorithm enables the agents to use uncoordinated step-sizes. We prove convergence of the local variable of each agent to the optimal solution of the aggregate optimization problem, without requiring decreasing step-sizes. In addition, we demonstrate the efficacy of our algorithm in distributed state estimation problems, and its robust counterparts, where we show its performance compared to existing distributed first-order optimization methods.
Paper Structure (13 sections, 4 theorems, 88 equations, 4 figures, 4 tables)

This paper contains 13 sections, 4 theorems, 88 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Notation and Preliminaries
  4. Problem Formulation And The Centralized Conjugate Gradient Method
  5. Distributed Conjugate Gradient Method
  6. Convergence Analysis
  7. Simulations
  8. Distributed State Estimation
  9. Distributed Robust Least-Squares
  10. Conclusion
  11. Proof of Lemma \ref{['lem:sequence_bounds']}
  12. Proof of Theorem \ref{['thm:agreement']}
  13. Proof of Theorem \ref{['thm:convergence']}

Key Result

Lemma 1

If the sequences ${\{\bm{x}^{(k)}\}_{\forall k}}$, ${\{\bm{s}^{(k)}\}_{\forall k}}$, and ${\{\bm{z}^{(k)}\}_{\forall k}}$ are generated by the recurrence in eq:x_sequence, eq:s_sequence, and eq:z_sequence, the auxiliary sequences ${\{\tilde{\bm{x}}^{(k)}\}_{\forall k}}$, ${\{\tilde{\bm{s}}^{(k)}\}_{

Figures (4)

  • Figure 1: Convergence error of all agents per iteration in the distributed state estimation problem on a fully-connected communication graph. DC-GRAD converges the fastest, closely followed by DIGing-ATC.
  • Figure 2: Convergence error of all agents per iteration in the distributed state estimation problem on a randomly-generated connected communication graph with ${\kappa = 0.48}$. C-ADMM attains the fastest convergence rate. The convergence plot of DIGing-ATC overlays that of DC-GRAD, with both algorithms converging at the same rate.
  • Figure 3: Convergence error of all agents per iteration in the distributed robust-state-estimation problem on a fully-connected communication network. DC-GRAD attains the fastest convergence rate, while $AB$/Push-Pull attains the slowest convergence rate.
  • Figure 4: Convergence error of all agents per iteration in the distributed robust-state-estimation problem on a randomly-generated connected communication network with ${\kappa = 0.42}$. The convergence plot of DIGing-ATC overlays that of DC-GRAD, with both methods converging faster than the other methods in this trial, although, in general, $ABm$ converges marginally faster on more-sparsely-connected graphs.

Theorems & Definitions (12)

  • Definition 1: Conjugacy
  • Definition 2: Convex Function
  • Definition 3: Smoothness
  • Definition 4: Coercive Function
  • Remark 1
  • Lemma 1
  • proof
  • Theorem 1: Agreement
  • proof
  • Theorem 2: Convergence of the Objective Value
  • ...and 2 more