Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OmniLRS: A Photorealistic Simulator for Lunar Robotics

Antoine Richard, Junnosuke Kamohara, Kentaro Uno, Shreya Santra, Dave van der Meer, Miguel Olivares-Mendez, Kazuya Yoshida

TL;DR

OmniLRS addresses the critical need for accessible, high-fidelity lunar robotics simulators by building a photorealistic environment on Nvidia IsaacSim that supports multi-robot operation and provides a complete synthetic data pipeline with ROS bindings. It combines real and procedurally generated terrains, photoreal rock assets, and domain randomization to enable perception training and sim-to-real evaluation, demonstrated through rock instance segmentation with YOLOv8. Key findings show that path-traced synthetic data yields the best transfer to real lunar imagery, and synthetic pretraining followed by real-data finetuning substantially improves performance, though transfer to real Apollo data remains challenging without richer lunar assets. The work delivers open-source tools, datasets, and demonstrations to accelerate perception and navigation research for lunar exploration, while highlighting areas for asset realism and terrain fidelity improvements to close the sim-to-real gap further.

Abstract

Developing algorithms for extra-terrestrial robotic exploration has always been challenging. Along with the complexity associated with these environments, one of the main issues remains the evaluation of said algorithms. With the regained interest in lunar exploration, there is also a demand for quality simulators that will enable the development of lunar robots. % In this paper, we explain how we built a Lunar simulator based on Isaac Sim, Nvidia's robotic simulator. In this paper, we propose Omniverse Lunar Robotic-Sim (OmniLRS) that is a photorealistic Lunar simulator based on Nvidia's robotic simulator. This simulation provides fast procedural environment generation, multi-robot capabilities, along with synthetic data pipeline for machine-learning applications. It comes with ROS1 and ROS2 bindings to control not only the robots, but also the environments. This work also performs sim-to-real rock instance segmentation to show the effectiveness of our simulator for image-based perception. Trained on our synthetic data, a yolov8 model achieves performance close to a model trained on real-world data, with 5% performance gap. When finetuned with real data, the model achieves 14% higher average precision than the model trained on real-world data, demonstrating our simulator's photorealism.% to realize sim-to-real. The code is fully open-source, accessible here: https://github.com/AntoineRichard/LunarSim, and comes with demonstrations.

OmniLRS: A Photorealistic Simulator for Lunar Robotics

TL;DR

OmniLRS addresses the critical need for accessible, high-fidelity lunar robotics simulators by building a photorealistic environment on Nvidia IsaacSim that supports multi-robot operation and provides a complete synthetic data pipeline with ROS bindings. It combines real and procedurally generated terrains, photoreal rock assets, and domain randomization to enable perception training and sim-to-real evaluation, demonstrated through rock instance segmentation with YOLOv8. Key findings show that path-traced synthetic data yields the best transfer to real lunar imagery, and synthetic pretraining followed by real-data finetuning substantially improves performance, though transfer to real Apollo data remains challenging without richer lunar assets. The work delivers open-source tools, datasets, and demonstrations to accelerate perception and navigation research for lunar exploration, while highlighting areas for asset realism and terrain fidelity improvements to close the sim-to-real gap further.

Abstract

Developing algorithms for extra-terrestrial robotic exploration has always been challenging. Along with the complexity associated with these environments, one of the main issues remains the evaluation of said algorithms. With the regained interest in lunar exploration, there is also a demand for quality simulators that will enable the development of lunar robots. % In this paper, we explain how we built a Lunar simulator based on Isaac Sim, Nvidia's robotic simulator. In this paper, we propose Omniverse Lunar Robotic-Sim (OmniLRS) that is a photorealistic Lunar simulator based on Nvidia's robotic simulator. This simulation provides fast procedural environment generation, multi-robot capabilities, along with synthetic data pipeline for machine-learning applications. It comes with ROS1 and ROS2 bindings to control not only the robots, but also the environments. This work also performs sim-to-real rock instance segmentation to show the effectiveness of our simulator for image-based perception. Trained on our synthetic data, a yolov8 model achieves performance close to a model trained on real-world data, with 5% performance gap. When finetuned with real data, the model achieves 14% higher average precision than the model trained on real-world data, demonstrating our simulator's photorealism.% to realize sim-to-real. The code is fully open-source, accessible here: https://github.com/AntoineRichard/LunarSim, and comes with demonstrations.
Paper Structure (29 sections, 4 figures, 1 table)

This paper contains 29 sections, 4 figures, 1 table.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. Lunar Simulators
  4. Dataset from Extraterrestrial Environment
  5. Method
  6. Terrain Generation and rendering
  7. In-Lab Acquisition
  8. Real Lunar Terrains
  9. Procedural Generation
  10. Importing Terrains in Isaac
  11. Lunar Environment Modeling
  12. Surface texture
  13. Rocks
  14. Procedural asset management
  15. Robot modelling
  16. ...and 14 more sections

Figures (4)

  • Figure 1: A render of Lunalab (left) and a real lunar terrain (right) using pathtraced rendering with OmniLRS. The environment includes three types of rover models: EX1, Leo rover, and Husky.
  • Figure 2: Left: One of the robot, EX1 delivered with the simulator demonstrating a Lunalab environment. Right: EX1 and a Leo rover in a larger scale environment (the real Lunar terrain).
  • Figure 3: Example images from both the real and simulated lunalab.
  • Figure 4: Qualitative results of instance segmentation on Lunalab test set (upper row) and Apollo mission images (lower row).