Communication Methods in Multi-Agent Reinforcement Learning
Christoph Wittner
TL;DR
This survey analyzes how communication mechanisms enable coordination in multi-agent reinforcement learning by systematically categorizing 29 publications into paradigms such as fully-connected message passing, implicit, attention-based, graph-based, and role-based/hierarchical approaches, including niche variants like delayed, language-grounded, and tacit communication. It finds no universal best framework; effectiveness and scalability depend on task structure and constraints, with attention- and graph-based methods offering favorable trade-offs in many scenarios. The work highlights critical gaps in standardized benchmarking of system-level metrics and robustness under imperfect communication, calling for rigorous evaluations to guide real-world deployment. Overall, the paper provides a taxonomy and comparative insights to help practitioners choose appropriate communication strategies and identifies key directions for robust, scalable MARL communication research.
Abstract
Multi-agent reinforcement learning is a promising research area that extends established reinforcement learning approaches to problems formulated as multi-agent systems. Recently, a multitude of communication methods have been introduced to this field to address problems such as partially observable environments, non-stationarity, and exponentially growing action spaces. Communication further enables efficient cooperation among all agents interacting in an environment. This work aims at providing an overview of communication techniques in multi-agent reinforcement learning. By an in-depth analysis of 29 publications on this topic, the strengths and weaknesses of explicit, implicit, attention-based, graph-based, and hierarchical/role-based communication are evaluated. The results of this comparison show that there is no general, optimal communication framework for every problem. On the contrary, the choice of communication depends heavily on the problem at hand. The comparison also highlights the importance of communication methods with low computational overhead to enable scalability to environments where many agents interact. Finally, the paper discusses current research gaps, emphasizing the need for standardized benchmarking of system-level metrics and improved robustness under realistic communication conditions to enhance the real-world applicability of these approaches.