JointLoc: A Real-time Visual Localization Framework for Planetary UAVs Based on Joint Relative and Absolute Pose Estimation

TL;DR

JointLoc addresses the challenge of real-time UAV localization in planetary environments lacking GNSS and distinctive landmarks by fusing absolute 2-DoF pose estimates with relative VO derived 6-DoF poses. It introduces a coarse-to-fine absolute localization module that uses local satellite maps, SuperPoint features, and LightGlue matching, combined with an adaptive area prediction mechanism and a pose fusion based local bundle adjustment. The framework achieves state-of-the-art accuracy and speed, notably 0.237 m RMSE at about 21 Hz, outperforming ORB-SLAM2/3 and image-matching baselines on simulated Mars terrains and real Ingenuity imagery. The work contributes a planetary UAV dataset and demonstrates a modular, real-time localization pipeline with adaptive confidence that is suitable for resource-constrained planetary missions.

Abstract

Unmanned aerial vehicles (UAVs) visual localization in planetary aims to estimate the absolute pose of the UAV in the world coordinate system through satellite maps and images captured by on-board cameras. However, since planetary scenes often lack significant landmarks and there are modal differences between satellite maps and UAV images, the accuracy and real-time performance of UAV positioning will be reduced. In order to accurately determine the position of the UAV in a planetary scene in the absence of the global navigation satellite system (GNSS), this paper proposes JointLoc, which estimates the real-time UAV position in the world coordinate system by adaptively fusing the absolute 2-degree-of-freedom (2-DoF) pose and the relative 6-degree-of-freedom (6-DoF) pose. Extensive comparative experiments were conducted on a proposed planetary UAV image cross-modal localization dataset, which contains three types of typical Martian topography generated via a simulation engine as well as real Martian UAV images from the Ingenuity helicopter. JointLoc achieved a root-mean-square error of 0.237m in the trajectories of up to 1,000m, compared to 0.594m and 0.557m for ORB-SLAM2 and ORB-SLAM3 respectively. The source code will be available at https://github.com/LuoXubo/JointLoc.

Figures (5)

• Figure 1: JointLoc accurately estimates the UAV poses. In contrast, the results of absolute localization are rough and pose a great challenge to the control algorithm. The results of relative localization are in the local coordinate system and require manual alignment to the world coordinate system.
• Figure 2: The pipeline of the proposed JointLoc consists of two threads: the absolute localization thread and the relative localization thread. In the relative localization thread, the UAV image is fed into visual odometry for relative pose estimation and keyframe selection. Simultaneously, the absolute localization module matches the UAV image with the candidate satellite map to estimate the absolute 2-DoF pose of the UAV. These absolute localization and visual odometry results are jointly processed by a pose fusion-based local bundle adjustment (LBA) algorithm. Then the transformation matrix from the relative coordinate system to the world coordinate system is calculated, resulting in the motion trajectory in the world coordinate system.
• Figure 3: LightGlue matches the feature points.
• Figure 4: Schematic diagram of the subsequent area prediction strategy for large-scale image localization. The confidence level from the last UAV localization is 0.7. Accordingly, a search radius of $\lceil 10^{1-0.7}\rceil=2$ is employed, with the local map containing the current location serving as the center. This search aims to identify the local map that offers the closest match to the current UAV image. Subsequently, with the confidence of the current positioning standing at 0.9, the search radius is reduced to 1 when seeking the future UAV position.
• Figure 5: Samples of the proposed planetary UAV localization datasets. The first two rows are simulated satellite maps and UAV images, and the latter two rows are Martian surface images taken by the Perseverance lander.