Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learning

Ziyu Cheng, Jinsheng Ren, Zhouxian Jiang, Chenzhihang Li, Rongye Shi, Bin Liang, Jun Yang

Abstract

Effective communication is pivotal for addressing complex collaborative tasks in multi-agent reinforcement learning (MARL). Yet, limited communication bandwidth and dynamic, intricate environmental topologies present significant challenges in identifying high-value communication partners. Agents must consequently select collaborators under uncertainty, lacking a priori knowledge of which partners can deliver task-critical information. To this end, we propose Interference-Aware K-Step Reachable Communication (IA-KRC), a novel framework that enhances cooperation via two core components: (1) a K-Step reachability protocol that confines message passing to physically accessible neighbors, and (2) an interference-prediction module that optimizes partner choice by minimizing interference while maximizing utility. Compared to existing methods, IA-KRC enables substantially more persistent and efficient cooperation despite environmental interference. Comprehensive evaluations confirm that IA-KRC achieves superior performance compared to state-of-the-art baselines, while demonstrating enhanced robustness and scalability in complex topological and highly dynamic multi-agent scenarios.

Interference-Aware K-Step Reachable Communication in Multi-Agent Reinforcement Learning

Abstract

Effective communication is pivotal for addressing complex collaborative tasks in multi-agent reinforcement learning (MARL). Yet, limited communication bandwidth and dynamic, intricate environmental topologies present significant challenges in identifying high-value communication partners. Agents must consequently select collaborators under uncertainty, lacking a priori knowledge of which partners can deliver task-critical information. To this end, we propose Interference-Aware K-Step Reachable Communication (IA-KRC), a novel framework that enhances cooperation via two core components: (1) a K-Step reachability protocol that confines message passing to physically accessible neighbors, and (2) an interference-prediction module that optimizes partner choice by minimizing interference while maximizing utility. Compared to existing methods, IA-KRC enables substantially more persistent and efficient cooperation despite environmental interference. Comprehensive evaluations confirm that IA-KRC achieves superior performance compared to state-of-the-art baselines, while demonstrating enhanced robustness and scalability in complex topological and highly dynamic multi-agent scenarios.
Paper Structure (32 sections, 14 equations, 13 figures, 4 tables, 6 algorithms)

This paper contains 32 sections, 14 equations, 13 figures, 4 tables, 6 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Method
  5. Computing $K$‑step reachability using multi-layer map
  6. Computing Cooperation Cost with an Interference Potential Field
  7. MARL with IA-KRC
  8. Experiments
  9. Comparative Performance in Complex Topologies
  10. Experimental Setup.
  11. Results and Analysis.
  12. Scaling and Structure Study of Multi-Agent Collaboration in IA-KRC
  13. Ablation Study of IA-KRC Mechanisms
  14. Generalization to Standard Obstacle-Free Environments
  15. Conclusion
  16. ...and 17 more sections

Figures (13)

  • Figure 1: Limitations of common neighborhood constraints in MARL communication. (a) Euclidean distance may misrepresent actual accessibility: agents A and B appear close (blue dashed line) but are separated by a long traversable path (yellow arrow); (b) Vision-based constraints capture physical proximity better but miss reachable agents hidden from view; (c) Even with direct visibility, hostile interference (red region) can block cooperation, forcing detours (yellow arrows).
  • Figure 2: Overview of IA-KRC framework for MARL communication. The K‑step Reachability Module restricts communication to agents within a physically reachable domain based on shortest transition distance. The Interference Prediction Module evaluates potential conflicts or adversarial effects, selecting low-interference partners. Combined, these modules enable dynamic grouping and robust leader–follower collaboration in complex environments.
  • Figure 2: Isolated-agent ratio (Iso Rate), mean algebraic connectivity ($\lambda_{2}$), and its variance (var) in Maze (12v12, 1000 groupings).
  • Figure 3: Illustration of the time-varying regulation (a) and the multi-layer map structure (b).
  • Figure 4: Custom SMACv2 maps. White regions denote obstacles.
  • ...and 8 more figures