PAGNet: Pluggable Adaptive Generative Networks for Information Completion in Multi-Agent Communication

TL;DR

PAGNet addresses the challenge of information-efficient coordination in partially observable cooperative MARL by integrating a pluggable adaptive generative network with information-level communication. It introduces an information-level weight network to selectively fuse local observations across agents and an adaptive information-completion network, guided by a global discriminator, to synthesize informative global states. A Transformer-based decoder replaces the standard MLP for value estimation, enabling effective extraction of sequential, weighted information, while the pluggable design supports offline pretraining and seamless integration with CTDE frameworks. Empirical results across Level-based Foraging, Hallway, and SMAC show competitive performance gains, interpretable communication patterns, and scalable information completion with manageable computational overhead. These contributions advance scalable, interpretable, and efficient MARL with communication for complex multi-agent tasks.

Abstract

For partially observable cooperative tasks, multi-agent systems must develop effective communication and understand the interplay among agents in order to achieve cooperative goals. However, existing multi-agent reinforcement learning (MARL) with communication methods lack evaluation metrics for information weights and information-level communication modeling. This causes agents to neglect the aggregation of multiple messages, thereby significantly reducing policy learning efficiency. In this paper, we propose pluggable adaptive generative networks (PAGNet), a novel framework that integrates generative models into MARL to enhance communication and decision-making. PAGNet enables agents to synthesize global states representations from weighted local observations and use these representations alongside learned communication weights for coordinated decision-making. This pluggable approach reduces the computational demands typically associated with the joint training of communication and policy networks. Extensive experimental evaluations across diverse benchmarks and communication scenarios demonstrate the significant performance improvements achieved by PAGNet. Furthermore, we analyze the emergent communication patterns and the quality of generated global states, providing insights into operational mechanisms.

Figures (9)

• Figure 1: The structure of PAGNet is differentiated by various colors representing different networks: green for the information-level weight network, purple for the adaptive generative network, blue for the mix network, and red for the Transformer-based decoder. (a) Information-level weight network. (b) Adaptive generative network. (c) The overall architecture. (d) Transformer-based decoder.
• Figure 2: Structure of information-level weight network. To provide a clearer demonstration, we set the number of agents to five and use the weight $W_t^5$ of Agent 5 as an example.
• Figure 3: Structure of adaptive generative network.
• Figure 4: (a) Illustrations of LBF tasks. (b) Average returns on LBF. (c) Illustrations of Hallway tasks. (d) Average battle win rate on Hallway.
• Figure 5: Performance comparisons with baselines on SMAC.