IFS: Information Flow Structure for Multi-agent Ad Hoc System

TL;DR

This work tackles open-system, non-stationary ad hoc teamwork by analyzing information flow in multi-agent systems. It introduces IFS, a framework combining a communication protocol for controlled agents (CPCA), a communication module (CM), and an information fusion framework (IFM/ISM) to enable variable-length data handling and robust teammate inference. Empirical results in StarCraft II SMAC show improved information flow, processing capacity, and cross-scenario generalization over baselines, validating the approach for complex NAHT tasks. The findings underscore the practical value of explicit information flow design for open, heterogeneous multi-agent collaboration.

Abstract

Multi-agent ad hoc systems are dynamic collaborative systems in which multiple autonomous agents must cooperate with both known and unknown teammates in open environments, without relying on pre-coordinated strategies. These systems operate under conditions of uncertainty and partial observability, where team composition, agent behaviors, and environmental factors may change during execution. Through an analysis of information flow in such systems, we identify two key limitations in existing research: insufficient information flow and limited information processing capacity. To address these issues, we propose an information flow structure for multi-agent ad hoc systems (IFS), which tackles these challenges from the perspectives of communication and information fusion. Experimental results in StarCraft II demonstrate that IFS significantly improves both information flow and processing capacity, while exhibiting strong generalization capabilities and outperforming baseline methods in complex ad hoc teamwork scenarios.

Figures (7)

• Figure 1: Training process of IFS. The actor network receives local observation data and local communication data. It fuses this information into a representation $e_i$ through an information fusion module (IFM), then updates the controlled agent's internal state $b_i$. This internal state $b_i$ is used by a communication module (CM) to generate the agent's communication data, which is broadcast to nearby controlled agents. Simultaneously, $b_i$ is used to generate the agent's current policy $\pi_i$. During the training phase, the representation $e_i$ is also processed by an information separation module (ISM) to separate specific information for use as a reconstruction error. Additionally, the critic network receives global information to produce an action-value estimate $q_t$ for the current global state.
• Figure 2: Communication module (CM). The yellow dashed circle represents the communication range of the controlled agent $c_i$. According to CPCA, $c_i$ can only communicate with other controlled agent $c_j$ within its communication range. It cannot communicate with uncontrolled agent $u_l$, nor with any agent $c_k$ located outside its communication range. The communication data $d_i$ of a controlled agent $c_i$ is derived from its internal state $b_i$, processed through a fully connected layer (FC) and layer normalization (LN).
• Figure 3: Information fusion module (IFM) and information separation module (ISM). The controlled agent $c_i$ receives local observation information $o_i$, which contains data from $k$ agents $ag_1, \ldots, ag_k$. Each of these data entries is processed independently through an FC layer and a ReLU layer. An attention mechanism (AM) is then applied to fuse these processed data into a single vector. This vector is subsequently passed through another FC layer and ReLU activation to generate the representation $e_i$. During the training phase, $e_i$ is further processed through an FC layer, ReLU activation, and another FC layer to produce the predicted data $o'_i$. This predicted data $o'_i$ encompasses the predicted information of all agents $ag'_1, \ldots, ag'_{|I|}$.
• Figure 4: Out-of-distribution (OOD) generalization testing. During training, the uncontrolled agent algorithms are selected from QMIX and IQL, while during testing, they are chosen from IPPO and VDN. Under general conditions, IFS demonstrates superior generalization performance compared to POAM, with this advantage being particularly pronounced in complex scenarios.
• Figure 5: Communication module testing. In the 8m_vs_9m scenario, we tested the performance of IFS with and without the CM. The testing CM experiment compares the average test returns and average ally deaths between the two configurations. Results show that the CM significantly enhances the ad hoc collaboration capability of controlled agents while reducing ally casualties. The dimensions of CM experiment examines the impact of varying communication dimensions, revealing that IFS exhibits low sensitivity to changes in this parameter. In the relationships between CM and OA experiment, we compared three information acquisition methods: communication-only (CM), observation of allies only (OA), and combined CM+OA. The findings demonstrate that CM alone is more effective than OA alone in improving cooperative performance.