Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Barzilai-Borwein Method for Distributed Optimization over Unbalanced Directed Networks

Jinhui Hu, Xin Chen, Lifeng Zheng, Ling Zhang, Huaqing Li

TL;DR

The paper addresses distributed convex optimization over unbalanced directed networks where agents hold private objectives. It introduces ADBB, a Barzilai-Borwein step-size based algorithm that uses multi-consensus inner loops and gradient tracking with only row-stochastic matrices to achieve accelerated linear convergence. The authors prove linear convergence under standard smoothness and strong convexity assumptions, extend BB step-size bounds to $\frac{1}{m\mu}$, and demonstrate practical gains in convergence speed and communication efficiency through numerical experiments. This work enables scalable, private, and fast distributed optimization in directed networks without requiring out-degree knowledge, offering a practical advancement for networked multi-agent systems.

Abstract

This paper studies optimization problems over multi-agent systems, in which all agents cooperatively minimize a global objective function expressed as a sum of local cost functions. Each agent in the systems uses only local computation and communication in the overall process without leaking their private information. Based on the Barzilai-Borwein (BB) method and multi-consensus inner loops, a distributed algorithm with the availability of larger stepsizes and accelerated convergence, namely ADBB, is proposed. Moreover, owing to employing only row-stochastic weight matrices, ADBB can resolve the optimization problems over unbalanced directed networks without requiring the knowledge of neighbors' out-degree for each agent. Via establishing contraction relationships between the consensus error, the optimality gap, and the gradient tracking error, ADBB is theoretically proved to converge linearly to the globally optimal solution. A real-world data set is used in simulations to validate the correctness of the theoretical analysis.

The Barzilai-Borwein Method for Distributed Optimization over Unbalanced Directed Networks

TL;DR

The paper addresses distributed convex optimization over unbalanced directed networks where agents hold private objectives. It introduces ADBB, a Barzilai-Borwein step-size based algorithm that uses multi-consensus inner loops and gradient tracking with only row-stochastic matrices to achieve accelerated linear convergence. The authors prove linear convergence under standard smoothness and strong convexity assumptions, extend BB step-size bounds to , and demonstrate practical gains in convergence speed and communication efficiency through numerical experiments. This work enables scalable, private, and fast distributed optimization in directed networks without requiring out-degree knowledge, offering a practical advancement for networked multi-agent systems.

Abstract

This paper studies optimization problems over multi-agent systems, in which all agents cooperatively minimize a global objective function expressed as a sum of local cost functions. Each agent in the systems uses only local computation and communication in the overall process without leaking their private information. Based on the Barzilai-Borwein (BB) method and multi-consensus inner loops, a distributed algorithm with the availability of larger stepsizes and accelerated convergence, namely ADBB, is proposed. Moreover, owing to employing only row-stochastic weight matrices, ADBB can resolve the optimization problems over unbalanced directed networks without requiring the knowledge of neighbors' out-degree for each agent. Via establishing contraction relationships between the consensus error, the optimality gap, and the gradient tracking error, ADBB is theoretically proved to converge linearly to the globally optimal solution. A real-world data set is used in simulations to validate the correctness of the theoretical analysis.
Paper Structure (19 sections, 13 theorems, 66 equations, 8 figures, 3 tables, 1 algorithm)

This paper contains 19 sections, 13 theorems, 66 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Motivations
  4. Contributions
  5. Organization
  6. Basic Notations
  7. Preliminaries
  8. Communication Network Model
  9. Problem Reformulation
  10. ADBB Development
  11. The Barzilai-Borwein Step-Sizes
  12. Distributed Optimization Using Only Row-Stochastic Weight Matrices
  13. ADBB for Distributed Optimization
  14. Convergence Analysis
  15. Auxiliary Relations
  16. ...and 4 more sections

Key Result

Lemma 1

Supposing that Assumptions A2-A3 hold, for $\forall k \ge 0$, the BB step-size $\alpha _k^i$, $i \in \mathcal{V}$ computed by (E6) or (E7) in ADBB satisfies

Figures (8)

  • Figure 1: An unbalanced directed network with 10 agents.
  • Figure 2: Performance of ADBB under different initial step-sizes $\alpha _0^i$.
  • Figure 3: Performance of ADBB under different multi-consensus inner loop numbers $H$.
  • Figure 4: An unbalanced directed network with 20 agents.
  • Figure 5: Performance comparison over gradients computation.
  • ...and 3 more figures

Theorems & Definitions (30)

  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Lemma 1
  • Proof 1
  • Remark 6
  • Lemma 2
  • Proof 2
  • ...and 20 more