Applications of Multi-Agent Deep Reinforcement Learning Communication in Network Management: A Survey
Yue Pi, Wang Zhang, Yong Zhang, Hairong Huang, Baoquan Rao, Yulong Ding, Shuanghua Yang
TL;DR
The paper surveys the use of multi-agent deep reinforcement learning (MADRL) with inter-agent communication for network management in autonomous driving networks, focusing on four application domains: traffic engineering, network spectrum access, transmit power control, and network security. It provides a structured analysis of communication schemes (fully centralized, CTDE, and fully distributed), message content (state, reward, mixed), communication objects (neighbor, all, central), processing methods (concatenation, averaging, neural fusion), and constraints (bandwidth and noise). The authors identify open issues—denoising, scalability, synchronization, emergent-communication readability, and security—and discuss future research directions to enable reliable, scalable, and secure MADRL-enabled ADN. This work offers a comprehensive framework for evaluating and designing agent communication in MADRL for modern, heterogeneous networks with autonomous control needs.
Abstract
With the advancement of artificial intelligence technology, the automation of network management, also known as Autonomous Driving Networks (ADN), is gaining widespread attention. The network management has shifted from traditional homogeneity and centralization to heterogeneity and decentralization. Multi-agent deep reinforcement learning (MADRL) allows agents to make decisions based on local observations independently. This approach is in line with the needs of automation and has garnered significant attention from academia and industry. In a distributed environment, information interaction between agents can effectively address the non-stationarity problem of multiple agents and promote cooperation. Therefore, in this survey, we first examined the application of MADRL in network management, including specific application fields such as traffic engineering, wireless network access, power control, and network security. Then, we conducted a detailed analysis of communication behavior between agents, including communication schemes, communication content construction, communication object selection, message processing, and communication constraints. Finally, we discussed the open issues and future research directions of agent communication in MADRL for future network management and ADN applications.