Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning

Weizheng Wang, Le Mao, Ruiqi Wang, Byung-Cheol Min

TL;DR

The paper tackles multi-robot socially-aware navigation (MR-SAN) in pedestrian-rich environments with limited communication by formulating the problem as a Dec-POSMDP and introducing SAMARL, a MARL framework augmented with a hybrid spatial-temporal transformer to model human-robot and robot-robot interactions. It combines a ST-graph social interaction encoder with MAPPO training to enable decentralized execution of cooperative, socially compliant policies that respect realistic kinematics. The key contributions include a dynamics-aware Dec-POSMDP formulation, a transformer-based HRI/RRI representation network, and extensive simulation and real-world demonstrations showing superior performance over baselines and ablations. The work advances practical, scalable multi-robot coordination in human environments, with potential impact on collaborative delivery, exploration, and service robotics in public spaces.

Abstract

In public spaces shared with humans, ensuring multi-robot systems navigate without collisions while respecting social norms is challenging, particularly with limited communication. Although current robot social navigation techniques leverage advances in reinforcement learning and deep learning, they frequently overlook robot dynamics in simulations, leading to a simulation-to-reality gap. In this paper, we bridge this gap by presenting a new multi-robot social navigation environment crafted using Dec-POSMDP and multi-agent reinforcement learning. Furthermore, we introduce SAMARL: a novel benchmark for cooperative multi-robot social navigation. SAMARL employs a unique spatial-temporal transformer combined with multi-agent reinforcement learning. This approach effectively captures the complex interactions between robots and humans, thus promoting cooperative tendencies in multi-robot systems. Our extensive experiments reveal that SAMARL outperforms existing baseline and ablation models in our designed environment. Demo videos for this work can be found at: https://sites.google.com/view/samarl

Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning

TL;DR

The paper tackles multi-robot socially-aware navigation (MR-SAN) in pedestrian-rich environments with limited communication by formulating the problem as a Dec-POSMDP and introducing SAMARL, a MARL framework augmented with a hybrid spatial-temporal transformer to model human-robot and robot-robot interactions. It combines a ST-graph social interaction encoder with MAPPO training to enable decentralized execution of cooperative, socially compliant policies that respect realistic kinematics. The key contributions include a dynamics-aware Dec-POSMDP formulation, a transformer-based HRI/RRI representation network, and extensive simulation and real-world demonstrations showing superior performance over baselines and ablations. The work advances practical, scalable multi-robot coordination in human environments, with potential impact on collaborative delivery, exploration, and service robotics in public spaces.

Abstract

In public spaces shared with humans, ensuring multi-robot systems navigate without collisions while respecting social norms is challenging, particularly with limited communication. Although current robot social navigation techniques leverage advances in reinforcement learning and deep learning, they frequently overlook robot dynamics in simulations, leading to a simulation-to-reality gap. In this paper, we bridge this gap by presenting a new multi-robot social navigation environment crafted using Dec-POSMDP and multi-agent reinforcement learning. Furthermore, we introduce SAMARL: a novel benchmark for cooperative multi-robot social navigation. SAMARL employs a unique spatial-temporal transformer combined with multi-agent reinforcement learning. This approach effectively captures the complex interactions between robots and humans, thus promoting cooperative tendencies in multi-robot systems. Our extensive experiments reveal that SAMARL outperforms existing baseline and ablation models in our designed environment. Demo videos for this work can be found at: https://sites.google.com/view/samarl
Paper Structure (17 sections, 11 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 17 sections, 11 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminary
  4. Kinematic & Dynamic Configuration
  5. Markov Decision Process Formulation
  6. Multi Robot Socially-aware Navigation Task Statement
  7. Methodology
  8. Social Interaction State Representation
  9. Multi-Agent Reinforcement Learning
  10. Experiments And Results
  11. Simulation Environment
  12. Baselines and Ablation Study
  13. Training Details
  14. Evaluation
  15. Results
  16. ...and 2 more sections

Figures (4)

  • Figure 1: An illustration of multi-robot cooperative socially-aware navigation: social robots are engaging in cooperative navigation while maintaining a safe social distance from pedestrians.
  • Figure 2: SAMARL Architecture: First, each robot feeds its individual local observations into the hybrid spatial-temporal transformer-based ST-graph social interaction encoder to create spatial-temporal state representations of HRI and RRI states. Then, the robot leverages environmental dynamics features to perform multi-robot cooperative navigation policies and adhere to social norm, using the MAPPO trainer and the social norm reward function within the multi robot strategy executor block. Finally, the generated macro-action (MA) and local-action (LA) guide the robots in the environment.
  • Figure 3: Learning curves of SAMARL and other two ablation models.
  • Figure 4: Comparison Trajectories Visualization: the trajectories visualization of ablation models and SAMARL that are tested by the same test case.