Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CAMAR: Continuous Actions Multi-Agent Routing

Artem Pshenitsyn, Aleksandr Panov, Alexey Skrynnik

TL;DR

CAMAR introduces a fast, GPU-accelerated benchmark for continuous-space multi-agent pathfinding, bridging MARL with planning approaches. It features force-based, circle-representation dynamics, diverse map generators, and LIDAR-inspired local observations, enabling scalable experiments with heterogeneous agents. A three-tier Easy/Medium/Hard evaluation protocol with IQM and CI95 supports rigorous generalization assessment, while benchmarks across six MARL algorithms and multiple hybrids demonstrate trade-offs between learning coordination and planning-based routing. CAMAR’s scalability analysis confirms throughput above 100k steps per second and effective operation with hundreds of agents, making it a practical, realistic testbed for large-scale MARL in continuous MAPF tasks. The environment’s modular design and planning integrations position CAMAR as a versatile platform for advancing both learning-based and planning-aware routing research.

Abstract

Multi-agent reinforcement learning (MARL) is a powerful paradigm for solving cooperative and competitive decision-making problems. While many MARL benchmarks have been proposed, few combine continuous state and action spaces with challenging coordination and planning tasks. We introduce CAMAR, a new MARL benchmark designed explicitly for multi-agent pathfinding in environments with continuous actions. CAMAR supports cooperative and competitive interactions between agents and runs efficiently at up to 100,000 environment steps per second. We also propose a three-tier evaluation protocol to better track algorithmic progress and enable deeper analysis of performance. In addition, CAMAR allows the integration of classical planning methods such as RRT and RRT* into MARL pipelines. We use them as standalone baselines and combine RRT* with popular MARL algorithms to create hybrid approaches. We provide a suite of test scenarios and benchmarking tools to ensure reproducibility and fair comparison. Experiments show that CAMAR presents a challenging and realistic testbed for the MARL community.

CAMAR: Continuous Actions Multi-Agent Routing

TL;DR

CAMAR introduces a fast, GPU-accelerated benchmark for continuous-space multi-agent pathfinding, bridging MARL with planning approaches. It features force-based, circle-representation dynamics, diverse map generators, and LIDAR-inspired local observations, enabling scalable experiments with heterogeneous agents. A three-tier Easy/Medium/Hard evaluation protocol with IQM and CI95 supports rigorous generalization assessment, while benchmarks across six MARL algorithms and multiple hybrids demonstrate trade-offs between learning coordination and planning-based routing. CAMAR’s scalability analysis confirms throughput above 100k steps per second and effective operation with hundreds of agents, making it a practical, realistic testbed for large-scale MARL in continuous MAPF tasks. The environment’s modular design and planning integrations position CAMAR as a versatile platform for advancing both learning-based and planning-aware routing research.

Abstract

Multi-agent reinforcement learning (MARL) is a powerful paradigm for solving cooperative and competitive decision-making problems. While many MARL benchmarks have been proposed, few combine continuous state and action spaces with challenging coordination and planning tasks. We introduce CAMAR, a new MARL benchmark designed explicitly for multi-agent pathfinding in environments with continuous actions. CAMAR supports cooperative and competitive interactions between agents and runs efficiently at up to 100,000 environment steps per second. We also propose a three-tier evaluation protocol to better track algorithmic progress and enable deeper analysis of performance. In addition, CAMAR allows the integration of classical planning methods such as RRT and RRT* into MARL pipelines. We use them as standalone baselines and combine RRT* with popular MARL algorithms to create hybrid approaches. We provide a suite of test scenarios and benchmarking tools to ensure reproducibility and fair comparison. Experiments show that CAMAR presents a challenging and realistic testbed for the MARL community.

Paper Structure

This paper contains 74 sections, 6 equations, 14 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. CAMAR Environment
  4. Dynamic Model & Action Space
  5. HolonomicDynamic
  6. DiffDriveDynamic
  7. Observations
  8. Circle-Based Discretization
  9. Map Generators
  10. Reward Function
  11. Heterogeneous agents
  12. Metrics
  13. Evaluation Protocols
  14. Experimental Evaluation
  15. Experimental Setup
  16. ...and 59 more sections

Figures (14)

  • Figure 1: An example scenario from the proposed CAMAR benchmark. Agents are represented as filled circles. Each agent aims to reach its goal while avoiding collisions. The small arrows for the red and green agents indicate segments of paths generated by RRT*, providing guidance for the RL algorithms.
  • Figure 3: LIDAR-inspired vector observations in CAMAR. Each agent detects nearby objects using penetration vectors, and receives a normalized goal direction.
  • Figure 4: A rich collection of maps for multi-agent planning in continuous spaces in CAMAR: support for both continuous and grid landscapes together with MovingAI collection sturtevant2012benchmarks.
  • Figure 5: Illustration of heterogeneous agents with different sizes and dynamics supported by CAMAR. Blue agents are governed by HolonomicDynamic, while green agents follow DiffDriveDynamic. All agents navigate a shared environment while avoiding gray obstacles.
  • Figure 6: Example of heterogeneous agent coordination (a). Success rates of algorithms are shown in (b).
  • ...and 9 more figures