Goal-Oriented Multi-Agent Reinforcement Learning for Decentralized Agent Teams

TL;DR

The paper targets coordination among multiple autonomous agents under limited communication and partial observability by proposing a fully decentralized MARL framework with goal-aware communication. Agents share learning weights only with peers that share the same individual goals, implemented via a weight-sharing update on actor and critic networks, and augmented with entropy-regularized action selection. Experiments in obstacle-rich grid environments show that goal-aware coordination (A5) improves task success rates and reduces time-to-goal, with maintained performance as the agent count scales, relative to non-collaborative baselines. The approach advances decentralized coordination for realistic multi-vehicle systems and points to future work on robustness, real-world testing, and domain-specific reward shaping.

Abstract

Connected and autonomous vehicles across land, water, and air must often operate in dynamic, unpredictable environments with limited communication, no centralized control, and partial observability. These real-world constraints pose significant challenges for coordination, particularly when vehicles pursue individual objectives. To address this, we propose a decentralized Multi-Agent Reinforcement Learning (MARL) framework that enables vehicles, acting as agents, to communicate selectively based on local goals and observations. This goal-aware communication strategy allows agents to share only relevant information, enhancing collaboration while respecting visibility limitations. We validate our approach in complex multi-agent navigation tasks featuring obstacles and dynamic agent populations. Results show that our method significantly improves task success rates and reduces time-to-goal compared to non-cooperative baselines. Moreover, task performance remains stable as the number of agents increases, demonstrating scalability. These findings highlight the potential of decentralized, goal-driven MARL to support effective coordination in realistic multi-vehicle systems operating across diverse domains.

Figures (6)

• Figure 1: The illustration of our coordination strategy. Agents begin at fixed positions. Agents 2 and 3 do not coordinate upon encountering each other due to differing goals. At step three, Agents 1 and 2 meet and coordinate, reaching their goals with four extra steps, while Agent 3, acting independently, takes 12 additional steps.
• Figure 2: Comparison between agent types in Scenario 1.
• Figure 3: Comparison between A1 and A5 in Scenario 2.
• Figure 4: Comparison between A1 and A5 in Scenario 3.
• Figure 5: An overview of the small environment for our experiments. In this scenario, A1 and A2 pursue G1, while A3 and A4 pursue G2.