Network Topology and Information Efficiency of Multi-Agent Systems: Study based on MARL

TL;DR

This paper addresses how network topology and information efficiency of communications influence MARL under non-stationarity and partial observability. It introduces a Directed Acyclic Graph (DAG) topology with learned order and depth, and two metrics—$IEI$ and $SEI$—to quantify message compactness and specialization. Empirical results on Grid World tasks show that directed and sequential communication improves coordination while reducing communication rounds, with deeper DAGs especially beneficial in heterogeneous settings; integrating the efficiency metrics into training speeds convergence and enhances success. Overall, the work demonstrates that adaptive communication topologies combined with information-aware messaging are essential for scalable, robust multi-agent coordination in complex environments.

Abstract

Multi-agent systems (MAS) solve complex problems through coordinated autonomous entities with individual decision-making capabilities. While Multi-Agent Reinforcement Learning (MARL) enables these agents to learn intelligent strategies, it faces challenges of non-stationarity and partial observability. Communications among agents offer a solution, but questions remain about its optimal structure and evaluation. This paper explores two underexamined aspects: communication topology and information efficiency. We demonstrate that directed and sequential topologies improve performance while reducing communication overhead across both homogeneous and heterogeneous tasks. Additionally, we introduce two metrics -- Information Entropy Efficiency Index (IEI) and Specialization Efficiency Index (SEI) -- to evaluate message compactness and role differentiation. Incorporating these metrics into training objectives improves success rates and convergence speed. Our findings highlight that designing adaptive communication topologies with information-efficient messaging is essential for effective coordination in complex MAS.

Figures (5)

• Figure 1: Illustration of the communications in MARL.
• Figure 2: General framework of the CTDE paradigm of the MARL with communications. (a) Centralized training, and (b) Decentralized execution.
• Figure 3: Illustration of the DAGs topology order and depth. The upper part shows the learned adjacency relationships between agents. The lower part demonstrates the resulting $3$ rounds of sequential communications among agents, derived from the DAG depth $d=3$ and sequential dependencies.
• Figure 4: Validating the effectiveness of the proposed DAG topology: Impacts of depth and order on performance (a) PP (b) PCP.
• Figure 5: Comparison of $\Phi_{\text{IEI}}$ and $\Phi_{\text{SEI}}$ for different algorithms in the TJ environment.