Think How Your Teammates Think: Active Inference Can Benefit Decentralized Execution

TL;DR

The paper addresses coordination gaps in decentralized MARL by removing reliance on explicit communication. It introduces AIM, a non-communication framework that models teammates' active inference through perception-belief-action portraits built from local observations, and blends these portraits with an attention-based relevance mechanism via a dual-filter to ensure accuracy and relevance. The approach is validated on SMAC, SMACv2, MPE, and GRF, showing superior or competitive performance and compatibility with multiple CTDE frameworks. This work advances robust, communication-free coordination in multi-agent systems with practical implications for noisy, delayed, or adversarial environments.

Abstract

In multi-agent systems, explicit cognition of teammates' decision logic serves as a critical factor in facilitating coordination. Communication (i.e., ``\textit{Tell}'') can assist in the cognitive development process by information dissemination, yet it is inevitably subject to real-world constraints such as noise, latency, and attacks. Therefore, building the understanding of teammates' decisions without communication remains challenging. To address this, we propose a novel non-communication MARL framework that realizes the construction of cognition through local observation-based modeling (i.e., \textit{``Think''}). Our framework enables agents to model teammates' \textbf{active inference} process. At first, the proposed method produces three teammate portraits: perception-belief-action. Specifically, we model the teammate's decision process as follows: 1) Perception: observing environments; 2) Belief: forming beliefs; 3) Action: making decisions. Then, we selectively integrate the belief portrait into the decision process based on the accuracy and relevance of the perception portrait. This enables the selection of cooperative teammates and facilitates effective collaboration. Extensive experiments on the SMAC, SMACv2, MPE, and GRF benchmarks demonstrate the superior performance of our method.

Figures (6)

• Figure 1: Modeling the Active Inference Process of the Teammate. In this scenario, agent $i$ models the perception-belief-action involved in its teammate's active inference process when facing the goalkeeper. This allows $i$ to obtain its teammate's decision-relevant information and achieve effective collaboration.
• Figure 2: The overall framework of AIM. (a) The training framework comprises the agent network and the mixing network; (b) The active inference module, which includes perception portrait, belief portrait, and action portrait; (c) The dual filter module, consisting of the accuracy filter and the relevance filter.
• Figure 3: Performance comparison between AIM and baselines on SMAC, SMACv2, and MPE. (a)-(f) Six representative maps on SMAC. (g)-(l) Six tasks on SMACv2. (m)-(o) Three tasks on MPE.
• Figure 4: Ablation studies. (a)-(b) illustrate module-wise ablation. (c)-(d) present loss-wise ablation. "AIM_w/o_Belief" removes belief portrait, "AIM_w/o_Action" removes action portrait, while "AIM_w/o_Filter" removes dual filter.
• Figure 5: Analysis of the parameter $k$ in selective collaboration. Different values of $k$ have varying impacts on performance.