Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OpenSocInt: A Multi-modal Training Environment for Human-Aware Social Navigation

Victor Sanchez, Chris Reinke, Ahamed Mohamed, Xavier Alameda-Pineda

TL;DR

This work tackles training autonomous agents for human-aware social navigation in multi-modal environments. It introduces OpenSocInt, a modular, open-source framework comprising a Simulator, Environment, and Agent to integrate diverse perceptual modalities (e.g., LEOG, RayCast, and closest-obstacle data) with multiple reinforcement learning algorithms (SAC, TD3, DDPG, A2C). An experimental protocol evaluates reward shaping, modality usage, and encoder fusion, revealing that pre-trained LEOG encoders, multi-modal fusion, and the SAC algorithm yield more data-efficient and stable learning, while different modalities influence early-stage performance. The framework enables systematic analysis of perception-to-action pipelines in social navigation, offering practical utility for researchers and developers and a path toward incorporating richer social cues in future work.

Abstract

In this paper, we introduce OpenSocInt, an open-source software package providing a simulator for multi-modal social interactions and a modular architecture to train social agents. We described the software package and showcased its interest via an experimental protocol based on the task of social navigation. Our framework allows for exploring the use of different perceptual features, their encoding and fusion, as well as the use of different agents. The software is already publicly available under GPL at https://gitlab.inria.fr/robotlearn/OpenSocInt/.

OpenSocInt: A Multi-modal Training Environment for Human-Aware Social Navigation

TL;DR

This work tackles training autonomous agents for human-aware social navigation in multi-modal environments. It introduces OpenSocInt, a modular, open-source framework comprising a Simulator, Environment, and Agent to integrate diverse perceptual modalities (e.g., LEOG, RayCast, and closest-obstacle data) with multiple reinforcement learning algorithms (SAC, TD3, DDPG, A2C). An experimental protocol evaluates reward shaping, modality usage, and encoder fusion, revealing that pre-trained LEOG encoders, multi-modal fusion, and the SAC algorithm yield more data-efficient and stable learning, while different modalities influence early-stage performance. The framework enables systematic analysis of perception-to-action pipelines in social navigation, offering practical utility for researchers and developers and a path toward incorporating richer social cues in future work.

Abstract

In this paper, we introduce OpenSocInt, an open-source software package providing a simulator for multi-modal social interactions and a modular architecture to train social agents. We described the software package and showcased its interest via an experimental protocol based on the task of social navigation. Our framework allows for exploring the use of different perceptual features, their encoding and fusion, as well as the use of different agents. The software is already publicly available under GPL at https://gitlab.inria.fr/robotlearn/OpenSocInt/.
Paper Structure (15 sections, 2 equations, 5 figures)

This paper contains 15 sections, 2 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Modular Architecture
  3. The Simulator Module
  4. The Environment Module
  5. The Agent Module
  6. Repository
  7. Experimental Protocol
  8. Reward functions
  9. Metrics
  10. Results
  11. The impact of the LEOG encoding
  12. The impact of the modalities
  13. The impact of fusing modalities
  14. The impact of the RL paradigm
  15. Conclusions

Figures (5)

  • Figure 1: Two examples of the environment simulation with OpenSocInt: (left) without and (right) with obstacles.
  • Figure 2: OpenSocInt's modular architecture: the agent takes actions and gets observations and rewards from the environment. Following these actions, the environment queries the simulator, and computes the reward and the observations from the simulated data.
  • Figure 3: Diagram of the goal force (left) and social/repulsion force (right) used in the simulator module to update the state of the simulation.
  • Figure 4: Architectures for processing the various input modalities. Convolutional layers (for LEOG) are in pink, fully connected layers in green, and radial basis fuction layers in orange. The circle stands for the concatenation-based multi-modal fusion. The various implemented agents always build from the latent space, shown in blue.
  • Figure 5: Mean and standard deviation of the success, collision, and truncated rates of the different settings evaluated. (a) Different LEOG encoding strategies: pre-trained frozen, pre-trained and fine-tuned, trained from scratch. (b) Different features: LEOG, RayCast and Closest Object. (c) Feature fusion: LEOG, RayCast, and LEOG+RayCast. (d) Various RL paradigms: SAC, TD3, DDPG, and A2C.