Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated Distributed Aggregative Optimization

Jiaxu Liu, Song Chen, Shengze Cai, Chao Xu, Jian Chu

TL;DR

This work tackles distributed aggregative optimization where each agent's local cost depends on its own state and a global aggregative term. It introduces two accelerated algorithms, DAGT-HB and DAGT-NES, that merge heavy-ball and Nesterov momentum with distributed aggregative gradient tracking to achieve global linear convergence under standard smoothness and strong convexity assumptions. Theoretical results establish explicit parameter regimes ensuring linear convergence, with corollaries translating to convergence of the objective value and state iterates; practical guidelines for parameter selection are provided. Numerical experiments on optimal placement and Nash-Cournot-style generation problems demonstrate faster convergence and robustness to topology, delays, and noise, highlighting the practical impact for scalable, accelerated distributed optimization in networked systems.

Abstract

This paper delves into the investigation of a distributed aggregative optimization problem within a network. In this scenario, each agent possesses its own local cost function, which relies not only on the local state variable but also on an aggregated function of state variables from all agents. To expedite the optimization process, we amalgamate the heavy ball and Nesterovs accelerated method with distributed aggregative gradient tracking, resulting in the proposal of two innovative algorithms, aimed at resolving the distributed aggregative optimization problem. Our analysis demonstrates that the proposed algorithms can converge to an optimal solution at a global linear convergence rate when the objective function is strongly convex with the Lipschitz-continuous gradient, and when the parameters (e.g., step size and momentum coefficients) are chosen within specific ranges. Additionally, we present several numerical experiments to verify the effectiveness, robustness and superiority of our proposed algorithms.

Accelerated Distributed Aggregative Optimization

TL;DR

This work tackles distributed aggregative optimization where each agent's local cost depends on its own state and a global aggregative term. It introduces two accelerated algorithms, DAGT-HB and DAGT-NES, that merge heavy-ball and Nesterov momentum with distributed aggregative gradient tracking to achieve global linear convergence under standard smoothness and strong convexity assumptions. Theoretical results establish explicit parameter regimes ensuring linear convergence, with corollaries translating to convergence of the objective value and state iterates; practical guidelines for parameter selection are provided. Numerical experiments on optimal placement and Nash-Cournot-style generation problems demonstrate faster convergence and robustness to topology, delays, and noise, highlighting the practical impact for scalable, accelerated distributed optimization in networked systems.

Abstract

This paper delves into the investigation of a distributed aggregative optimization problem within a network. In this scenario, each agent possesses its own local cost function, which relies not only on the local state variable but also on an aggregated function of state variables from all agents. To expedite the optimization process, we amalgamate the heavy ball and Nesterovs accelerated method with distributed aggregative gradient tracking, resulting in the proposal of two innovative algorithms, aimed at resolving the distributed aggregative optimization problem. Our analysis demonstrates that the proposed algorithms can converge to an optimal solution at a global linear convergence rate when the objective function is strongly convex with the Lipschitz-continuous gradient, and when the parameters (e.g., step size and momentum coefficients) are chosen within specific ranges. Additionally, we present several numerical experiments to verify the effectiveness, robustness and superiority of our proposed algorithms.

Paper Structure

This paper contains 22 sections, 20 theorems, 133 equations, 9 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries And Problem Setup
  3. Basic Notations
  4. Graph Theory
  5. Problem Formulations
  6. DAGT-HB
  7. Auxiliary Results
  8. Main Result
  9. DAGT-NES
  10. Auxiliary Results
  11. Main Result
  12. Discussion
  13. Numerical Simulation
  14. Simulation 1: The Optimal Placement Problem
  15. Simulation 2: A Class of Nash-Cournot Game of Generation Systems chen2024achieving
  16. ...and 7 more sections

Key Result

Lemma 1

Li2022Aggregative Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be $\mu$-strongly convex and $L$-smooth. When $\alpha \in(0,1 / L]$, then for all $x, y \in \mathbb{R}^n$, we can obtain $\|x-\alpha \nabla f(x)-(y-\alpha \nabla f(y))\| \leq(1-\mu \alpha)\|x-y\|$.

Figures (9)

  • Figure 1: (a): The evolution of the first coordinate of $x_{i,k}$ by using DAGT-HB. (b): The evolution of the second coordinate of $x_{i,k}$ by using DAGT-HB. (c): The evolution of the first coordinate of $u_{i,k}$ by using DAGT-HB. (d): The evolution of the second coordinate of $u_{i,k}$ by using DAGT-HB.
  • Figure 2: (a): The evolution of the first coordinate of $x_{i,k}$ by using DAGT-NES. (b): The evolution of the second coordinate of $x_{i,k}$ by using DAGT-NES. (c): The evolution of the first coordinate of $u_{i,k}$ by using DAGT-NES. (d): The evolution of the second coordinate of $u_{i,k}$ by using DAGT-NES.
  • Figure 3: The state error $\frac{\sum_{i=1}^5\|x_{i,k}-x_i^*\|^2}{5}$ comparison among DAGT, DAGT-HB, DAGT-NES and A-DAGT.
  • Figure 4: The relative error trajectories of $(F(x_k)-F(x^*))^2$ under different parameters.
  • Figure 5: Left: The evolution of $x_{i,k}$ by using DAGT-HB. Right: The evolution of $x_{i,k}$ by using DAGT-NES.
  • ...and 4 more figures

Theorems & Definitions (43)

  • Definition 1
  • Definition 2
  • Remark 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • Proof
  • ...and 33 more