Vision-based Geo-Localization of Future Mars Rotorcraft in Challenging Illumination Conditions

TL;DR

Map-based localization is essential for drift-free navigation of Mars rotorcraft in GNSS-denied environments. The authors introduce Geo-LoFTR, a geometry-aware transformer-based image matcher, and MARTIAN, a Blender-based Mars rendering tool, to generate realistic training data and evaluate MbL under diverse illumination and scale conditions. Geo-LoFTR fuses depth-derived geometry with image features to produce robust matches, yielding superior localization accuracy across sun elevation/azimuth changes, altitude variations, and simulated Martian days. This approach broadens the operational envelope of autonomous Mars aerial missions by enabling reliable online geo-localization in challenging lighting scenarios.

Abstract

Planetary exploration using aerial assets has the potential for unprecedented scientific discoveries on Mars. While NASA's Mars helicopter Ingenuity proved flight in Martian atmosphere is possible, future Mars rotocrafts will require advanced navigation capabilities for long-range flights. One such critical capability is Map-based Localization (MbL) which registers an onboard image to a reference map during flight in order to mitigate cumulative drift from visual odometry. However, significant illumination differences between rotocraft observations and a reference map prove challenging for traditional MbL systems, restricting the operational window of the vehicle. In this work, we investigate a new MbL system and propose Geo-LoFTR, a geometry-aided deep learning model for image registration that is more robust under large illumination differences than prior models. The system is supported by a custom simulation framework that uses real orbital maps to produce large amounts of realistic images of the Martian terrain. Comprehensive evaluations show that our proposed system outperforms prior MbL efforts in terms of localization accuracy under significant lighting and scale variations. Furthermore, we demonstrate the validity of our approach across a simulated Martian day.

Figures (13)

• Figure 1: Given an ortho-projected map of the terrain and a simulated onboard image we aim to estimate the pose of a rotocraft operating on Mars. Assuming a noisy pose prior, a search area is selected that is further divided into smaller regions and passed sequentially to our geometrically-enhanced Geo-LoFTR observation-to-map matcher. Geo-LoFTR is trained from data generated by our simulation framework MARTIAN. Finally, the matches are then filtered and passed to RANSAC-PnP to estimate the pose.
• Figure 2: Architecture of Geo-LoFTR that uses as inputs the query image $I^A$ and a map crop image $I^B$ along with its corresponding depth $I^C$. Geo-LoFTR learns to merge visual information from $I^B$ with geometric from $I^C$ using parallel $CrossAttn$ operations. The produced features then follow the coarse-to-fine approach proposed in LoFTR loftr. Our experimental evaluation shows that Geo-LoFTR is more robust than the original LoFTR under challenging illumination conditions.
• Figure 3: View of the Jezero Crater's DTM in MARTIAN.
• Figure 4: Gray scale images of a map tile from the Jezero crater site, rendered with different combinations of sun AZ and EL. Generated with MARTIAN.
• Figure 5: Tiles from orthographic maps at sun (AZ=0°, EL=5°) (left) (AZ=180°, EL=40°) (center) with three sampled query observations (right).