Advancing Behavior Generation in Mobile Robotics through High-Fidelity Procedural Simulations

TL;DR

YamaS addresses the reality gap in mobile robotics simulation by integrating Unity3D with ROS1/ROS2 to create high-fidelity, procedurally generated environments for single- and multi-agent navigation. It combines NLP-driven environment descriptions, VR-enabled human-robot interaction, and a dual-ROS bridge to support Deep-RL and language-based planning, validated against the Yamabiko Beego robot. The paper demonstrates precise sensor and kinematic fidelity, supported by a exacting motion model and real-world odometry comparison, while enabling LLM-driven reasoning and task execution in VR-enhanced scenarios. Overall, YamaS provides a versatile, scalable platform for rapid development, testing, and transfer of autonomous navigation strategies to real-world robotics, with potential to accelerate learning and collaboration in multi-agent and HRI contexts.

Abstract

This paper introduces YamaS, a simulator integrating Unity3D Engine with Robotic Operating System for robot navigation research and aims to facilitate the development of both Deep Reinforcement Learning (Deep-RL) and Natural Language Processing (NLP). It supports single and multi-agent configurations with features like procedural environment generation, RGB vision, and dynamic obstacle navigation. Unique to YamaS is its ability to construct single and multi-agent environments, as well as generating agent's behaviour through textual descriptions. The simulator's fidelity is underscored by comparisons with the real-world Yamabiko Beego robot, demonstrating high accuracy in sensor simulations and spatial reasoning. Moreover, YamaS integrates Virtual Reality (VR) to augment Human-Robot Interaction (HRI) studies, providing an immersive platform for developers and researchers. This fusion establishes YamaS as a versatile and valuable tool for the development and testing of autonomous systems, contributing to the fields of robot simulation and AI-driven training methodologies.

Figures (6)

• Figure 1: Interplay between Reality and Simulation: The Yamabiko Beego robot in action alongside its simulated avatar, with a researcher in VR guiding the task for behavior generation. This visual encapsulates the seamless real-time integration achieved through ROS1 and ROS2, pivotal for synchronizing the physical and virtual realms to refine autonomous navigation strategies.
• Figure 2: Architecture diagram showcasing the integration of the Unity Engine with ROS for the Yamabiko Beego robot simulation. This setup enables precise management and control of robotic behaviors and environmental dynamics through Python3 scripts, illustrating the seamless data exchange facilitated by the ROS Bridge between virtual and real-world components.
• Figure 3: Example of a procedurally generated environments within the YamaS simulator. This visualization highlights the versatility of our simulation framework, where altering parameters via ROS topics dynamically alters the scene, introducing new challenges and scenarios for agent navigation and task execution.
• Figure 4: Flow diagram and visualization of YamaS's procedural environment setup for multi-agent simulations, showcasing parameter selection for areas, robots, and obstacles, leading to an interactive grid generation. The process culminates in a fully-formed simulation space, demonstrating the environment's readiness for multi-agent collaboration and task execution.
• Figure 5: Comparative real-time odometry of the Yamabiko robot showcasing the pentagon path followed by the real robot (in red) against the desired trajectory (dashed line) and the simulated path (in blue), demonstrating the YamaS simulator's precision in mimicking actual robotic movement.