Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Context-aware Mamba-based Reinforcement Learning for social robot navigation

Syed Muhammad Mustafa, Omema Rizvi, Zain Ahmed Usmani, Abdul Basit Memon, Muhammad Mobeen Movania

TL;DR

This work introduces a new SRN planner, showcasing the potential for deep-state space models for robot navigation and CAMRL (Context-Aware Mamba-based Reinforcement Learning), a new deep learning-based State Space Model that has achieved results comparable to transformers in sequencing tasks.

Abstract

Social robot navigation (SRN) is a relevant problem that involves navigating a pedestrian-rich environment in a socially acceptable manner. It is an essential part of making social robots effective in pedestrian-rich settings. The use cases of such robots could vary from companion robots to warehouse robots to autonomous wheelchairs. In recent years, deep reinforcement learning has been increasingly used in research on social robot navigation. Our work introduces CAMRL (Context-Aware Mamba-based Reinforcement Learning). Mamba is a new deep learning-based State Space Model (SSM) that has achieved results comparable to transformers in sequencing tasks. CAMRL uses Mamba to determine the robot's next action, which maximizes the value of the next state predicted by the neural network, enabling the robot to navigate effectively based on the rewards assigned. We evaluate CAMRL alongside existing solutions (CADRL, LSTM-RL, SARL) using a rigorous testing dataset which involves a variety of densities and environment behaviors based on ORCA and SFM, thus, demonstrating that CAMRL achieves higher success rates, minimizes collisions, and maintains safer distances from pedestrians. This work introduces a new SRN planner, showcasing the potential for deep-state space models for robot navigation.

Context-aware Mamba-based Reinforcement Learning for social robot navigation

TL;DR

This work introduces a new SRN planner, showcasing the potential for deep-state space models for robot navigation and CAMRL (Context-Aware Mamba-based Reinforcement Learning), a new deep learning-based State Space Model that has achieved results comparable to transformers in sequencing tasks.

Abstract

Social robot navigation (SRN) is a relevant problem that involves navigating a pedestrian-rich environment in a socially acceptable manner. It is an essential part of making social robots effective in pedestrian-rich settings. The use cases of such robots could vary from companion robots to warehouse robots to autonomous wheelchairs. In recent years, deep reinforcement learning has been increasingly used in research on social robot navigation. Our work introduces CAMRL (Context-Aware Mamba-based Reinforcement Learning). Mamba is a new deep learning-based State Space Model (SSM) that has achieved results comparable to transformers in sequencing tasks. CAMRL uses Mamba to determine the robot's next action, which maximizes the value of the next state predicted by the neural network, enabling the robot to navigate effectively based on the rewards assigned. We evaluate CAMRL alongside existing solutions (CADRL, LSTM-RL, SARL) using a rigorous testing dataset which involves a variety of densities and environment behaviors based on ORCA and SFM, thus, demonstrating that CAMRL achieves higher success rates, minimizes collisions, and maintains safer distances from pedestrians. This work introduces a new SRN planner, showcasing the potential for deep-state space models for robot navigation.
Paper Structure (21 sections, 6 equations, 4 figures, 2 tables, 1 algorithm)

This paper contains 21 sections, 6 equations, 4 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. The collision avoidance paradigm
  3. Related Work
  4. DRL models for collision avoidance
  5. State Space Models
  6. State Space Models in Learning
  7. Selective State Space Model (Mamba)
  8. State Space Models in the Context of Reinforcement Learning
  9. Methodology
  10. Preliminaries
  11. Deep V-learning
  12. Reward Function
  13. State space models in the context of reinforcement learning
  14. Context Aware Mamba-based Reinforcement Learning
  15. Experimental Setup
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Framework for collision avoidance using deep reinforcement learning
  • Figure 2: Architecture for CAMRL
  • Figure 3: Diverse-4 Testing Scenarios from Sigal2023.
  • Figure 4: Comparison of our model with state-of-the-art models on the basis of success, timeout and collision rate