Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model-Free Dynamic Consensus in Multi-Agent Systems: A Q-Function Perspective

Maryam Babazadeh, Naim Bajcinca

TL;DR

The paper addresses dynamic consensus in linear discrete-time MAS with potentially marginally stable or unstable agent dynamics, where feasibility hinges on the graph Laplacian spectrum and a prescribed convergence rate. It introduces a Q-function–inspired reformulation of the LQR problem, recasting consensus design as an SDP with LMIs to jointly design the local feedback $K$ and coupling gains $c$, and develops a convex–concave decomposition to guarantee convergence to a stationary point of the nonconvex feasibility problem. A model-free extension uses data-driven SDP formulations based on trajectory data with persistent excitation, achieving equivalent performance without explicit system identification. An iterative algorithm balances feasibility, rate, robustness, and energy efficiency, and simulations on small and large networks validate the approach and illustrate practical scalability. The framework offers a robust, scalable method for distributed dynamic consensus in both directed and undirected graphs and points to future work on heterogeneous agents.

Abstract

This paper presents a new method for achieving dynamic consensus in linear discrete-time homogeneous multi-agent systems (MAS) with marginally stable or unstable dynamics. The guarantee of consensus in this setting involves a set of constraints based on the graph's spectral properties, complicating the design of the coupling gains. This challenge intensifies for large-scale systems with diverse graph Laplacian spectra. The proposed approach reformulates the dynamic consensus problem with a prescribed convergence rate using a state-action value function framework inspired by optimal control theory. Specifically, a synthetic linear quadratic regulation (LQR) formulation is introduced to encode the consensus objective, enabling its translation into a convex semidefinite programming (SDP) problem. The resulting SDP is applicable in both model-based and model-free settings for jointly designing the local feedback and coupling gains. To handle the inherent non-convex feasibility conditions, a convex-concave decomposition strategy is employed. Adaptation of the method in a completely model-free set-up eliminates the need for system identification or knowledge of the agents' dynamics. Instead, it relies on input-state data collection and offers an entirely data-driven equivalent SDP formulation. Finally, a new algorithm balancing feasibility, convergence rate, robustness, and energy efficiency, is established to provide design flexibility. Numerical simulations validate the method's effectiveness in various scenarios.

Model-Free Dynamic Consensus in Multi-Agent Systems: A Q-Function Perspective

TL;DR

The paper addresses dynamic consensus in linear discrete-time MAS with potentially marginally stable or unstable agent dynamics, where feasibility hinges on the graph Laplacian spectrum and a prescribed convergence rate. It introduces a Q-function–inspired reformulation of the LQR problem, recasting consensus design as an SDP with LMIs to jointly design the local feedback and coupling gains , and develops a convex–concave decomposition to guarantee convergence to a stationary point of the nonconvex feasibility problem. A model-free extension uses data-driven SDP formulations based on trajectory data with persistent excitation, achieving equivalent performance without explicit system identification. An iterative algorithm balances feasibility, rate, robustness, and energy efficiency, and simulations on small and large networks validate the approach and illustrate practical scalability. The framework offers a robust, scalable method for distributed dynamic consensus in both directed and undirected graphs and points to future work on heterogeneous agents.

Abstract

This paper presents a new method for achieving dynamic consensus in linear discrete-time homogeneous multi-agent systems (MAS) with marginally stable or unstable dynamics. The guarantee of consensus in this setting involves a set of constraints based on the graph's spectral properties, complicating the design of the coupling gains. This challenge intensifies for large-scale systems with diverse graph Laplacian spectra. The proposed approach reformulates the dynamic consensus problem with a prescribed convergence rate using a state-action value function framework inspired by optimal control theory. Specifically, a synthetic linear quadratic regulation (LQR) formulation is introduced to encode the consensus objective, enabling its translation into a convex semidefinite programming (SDP) problem. The resulting SDP is applicable in both model-based and model-free settings for jointly designing the local feedback and coupling gains. To handle the inherent non-convex feasibility conditions, a convex-concave decomposition strategy is employed. Adaptation of the method in a completely model-free set-up eliminates the need for system identification or knowledge of the agents' dynamics. Instead, it relies on input-state data collection and offers an entirely data-driven equivalent SDP formulation. Finally, a new algorithm balancing feasibility, convergence rate, robustness, and energy efficiency, is established to provide design flexibility. Numerical simulations validate the method's effectiveness in various scenarios.

Paper Structure

This paper contains 7 sections, 8 theorems, 75 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Preliminaries
  4. Model-based Dynamic Consensus Analysis and Design
  5. The Proposed Model-free Consensus Algorithm
  6. Simulation Results
  7. Conclusion

Key Result

Theorem 1

(bertsekas2012dynamic) Consider the Riccati equation with $k = 0, 1, \ldots,$ where the initial matrix $P_0$ is an arbitrary positive semidefinite symmetric matrix. Assume that the pair $(A, B)$ is controllable and $Q = C^T C$, where the pair $(A, C)$ is observable. Then, there exists $P \in \mathbb{S}^n_{++}$ such that for every $P_0 \in \mathbb{S} within the class of positive semi-definite ma

Figures (4)

  • Figure 1: The communication graph in Example 1.
  • Figure 2: The communication graph in Example 2.
  • Figure 3: Dynamic consensus in Example 1 with (a) State $1$ of all the agents, (b) State $2$ of all the agents, and (c) State $3$ of all the agents.
  • Figure 4: Dynamic consensus of 25 agents in Example 2 with (a) State $1$ of all the agents, (b) State $2$ of all the agents, and (c) State $3$ of all the agents.

Theorems & Definitions (11)

  • Theorem 1
  • Theorem 2
  • Lemma 1
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Remark 1
  • Definition 1
  • Theorem 6
  • Remark 2
  • ...and 1 more