LuSeg: Efficient Negative and Positive Obstacles Segmentation via Contrast-Driven Multi-Modal Feature Fusion on the Lunar

TL;DR

This work addresses autonomous obstacle segmentation for lunar rovers by introducing a high-fidelity Lunar Exploration Simulator System (LESS) and the LunarSeg RGB-D dataset containing both positive and negative obstacles. It then proposes LuSeg, a two-stage network that uses a Contrast-Driven Fusion Module (CDFM) to align semantic features across RGB and depth modalities during training, while keeping inference overhead minimal. The approach delivers state-of-the-art performance on LunarSeg and a real-world NPO dataset, achieving real-time speeds around 57 Hz. The authors release the LESS system, LunarSeg dataset, and LuSeg code to foster further research in extraterrestrial perception and navigation.

Abstract

As lunar exploration missions grow increasingly complex, ensuring safe and autonomous rover-based surface exploration has become one of the key challenges in lunar exploration tasks. In this work, we have developed a lunar surface simulation system called the Lunar Exploration Simulator System (LESS) and the LunarSeg dataset, which provides RGB-D data for lunar obstacle segmentation that includes both positive and negative obstacles. Additionally, we propose a novel two-stage segmentation network called LuSeg. Through contrastive learning, it enforces semantic consistency between the RGB encoder from Stage I and the depth encoder from Stage II. Experimental results on our proposed LunarSeg dataset and additional public real-world NPO road obstacle dataset demonstrate that LuSeg achieves state-of-the-art segmentation performance for both positive and negative obstacles while maintaining a high inference speed of approximately 57\,Hz. We have released the implementation of our LESS system, LunarSeg dataset, and the code of LuSeg at:https://github.com/nubot-nudt/LuSeg.

Figures (8)

• Figure 1: The scene of a lunar rover driving safely and autonomously in the Lunar Exploration Simulator System
• Figure 2: Overall architecture of our proposed LuSeg. There are two stages: Stage I involves single-modal training using only RGB images as input, while Stage II focuses on multi-modal training with both RGB and depth images as input. In Stage II, the output of the depth encoder is aligned with the output of the RGB encoder from Stage I, whose parameters are frozen during this stage. This serves as input to our proposed Contrast-Driven Fusion Module (CDFM). The final output of Stage II is the result of our LuSeg.
• Figure 3: The Overall of Lunar Exploration Simulator System(LESS).
• Figure 4: Various lunar surface scenes contained in the LunarSeg dataset. (a) high illumination and flat terrain (HF), (b) high illumination and rough terrain (HR), (c) low illumination and flat terrain (LF), and (d) low illumination and rough terrain (LR).
• Figure 5: Sample RGB images, depth images, and ground truth in our LunarSeg dataset. and represent negative obstacles and positive obstacles, respectively.