Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust and Efficient Communication in Multi-Agent Reinforcement Learning

Zejiao Liu, Yi Li, Jiali Wang, Junqi Tu, Yitian Hong, Fangfei Li, Yang Liu, Toshiharu Sugawara, Yang Tang

TL;DR

This survey addresses the gap between theoretical MARL models and real-world deployments by focusing on robust and bandwidth-efficient communication under non-ideal conditions, such as perturbations, delays, and bandwidth limits. It reviews foundational problem representations (Dec-POMDPs and Markov games), then surveys robustness against observation and message perturbations, along with delay-aware and bandwidth-aware learning frameworks. The authors highlight three key application domains—cooperative autonomous driving, distributed SLAM, and federated learning—to illustrate practical challenges and solutions, including adversarial defenses, information bottlenecks, and dynamic scheduling. The work advocates a unified, co-design approach across communication, learning, and robustness to bridge theory and practice, and outlines open challenges in security, delays, cross-layer optimization, and the use of large models for interpretable communication. The practical impact lies in guiding researchers toward designing MARL systems that perform reliably and efficiently in real-world, bandwidth-constrained, and adversarial environments.

Abstract

Multi-agent reinforcement learning (MARL) has made significant strides in enabling coordinated behaviors among autonomous agents. However, most existing approaches assume that communication is instantaneous, reliable, and has unlimited bandwidth; these conditions are rarely met in real-world deployments. This survey systematically reviews recent advances in robust and efficient communication strategies for MARL under realistic constraints, including message perturbations, transmission delays, and limited bandwidth. Furthermore, because the challenges of low-latency reliability, bandwidth-intensive data sharing, and communication-privacy trade-offs are central to practical MARL systems, we focus on three applications involving cooperative autonomous driving, distributed simultaneous localization and mapping, and federated learning. Finally, we identify key open challenges and future research directions, advocating a unified approach that co-designs communication, learning, and robustness to bridge the gap between theoretical MARL models and practical implementations.

Robust and Efficient Communication in Multi-Agent Reinforcement Learning

TL;DR

This survey addresses the gap between theoretical MARL models and real-world deployments by focusing on robust and bandwidth-efficient communication under non-ideal conditions, such as perturbations, delays, and bandwidth limits. It reviews foundational problem representations (Dec-POMDPs and Markov games), then surveys robustness against observation and message perturbations, along with delay-aware and bandwidth-aware learning frameworks. The authors highlight three key application domains—cooperative autonomous driving, distributed SLAM, and federated learning—to illustrate practical challenges and solutions, including adversarial defenses, information bottlenecks, and dynamic scheduling. The work advocates a unified, co-design approach across communication, learning, and robustness to bridge theory and practice, and outlines open challenges in security, delays, cross-layer optimization, and the use of large models for interpretable communication. The practical impact lies in guiding researchers toward designing MARL systems that perform reliably and efficiently in real-world, bandwidth-constrained, and adversarial environments.

Abstract

Multi-agent reinforcement learning (MARL) has made significant strides in enabling coordinated behaviors among autonomous agents. However, most existing approaches assume that communication is instantaneous, reliable, and has unlimited bandwidth; these conditions are rarely met in real-world deployments. This survey systematically reviews recent advances in robust and efficient communication strategies for MARL under realistic constraints, including message perturbations, transmission delays, and limited bandwidth. Furthermore, because the challenges of low-latency reliability, bandwidth-intensive data sharing, and communication-privacy trade-offs are central to practical MARL systems, we focus on three applications involving cooperative autonomous driving, distributed simultaneous localization and mapping, and federated learning. Finally, we identify key open challenges and future research directions, advocating a unified approach that co-designs communication, learning, and robustness to bridge the gap between theoretical MARL models and practical implementations.

Paper Structure

This paper contains 29 sections, 17 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Multi-agent problem representations
  3. Markov games
  4. Decentralized partially observable Markov decision process
  5. Robustness and Bandwidth: Addressing Communication Challenges in MARL
  6. Robust MARL with unreliable inputs and communication
  7. Perturbations on observations/states
  8. Perturbations on messages
  9. Other perturbations
  10. Limited bandwidth communication constraints
  11. MARL with Communication Delays
  12. Fixed delay and asynchronous information fusion methods
  13. Delay-aware mechanism and communication robustness
  14. Optimizing Communication Efficiency
  15. Message-level compression and simplification
  16. ...and 14 more sections

Figures (4)

  • Figure 1: Robust communication and cooperation in MASs: Challenges and implications.
  • Figure 2: Taxonomy of communication strategy optimization in MARL, structured along three key dimensions: when to transmit, whom/how to communicate, and what/rate to transmit. Robustness-oriented methods are highlighted separately.
  • Figure 3: Information integration paradigms in MARL communication. Methods evolve from naive pooling to richer representation learning, graph-based structures, information-theoretic formulations, self-supervised objectives, and driven by the goals of expressiveness, cost reduction, robustness, and scalability.
  • Figure 4: Representative application domains of communication-efficient MARL, highlighting distinct bandwidth/latency constraints, message semantics, robustness issues, and scheduling requirements in cooperative driving, distributed SLAM, and federated MARL.