UASTHN: Uncertainty-Aware Deep Homography Estimation for UAV Satellite-Thermal Geo-localization

TL;DR

UASTHN tackles uncertainty in thermal geo-localization for UAVs by introducing CropTTA, a data-uncertainty-aware test-time augmentation, and Deep Ensembles to quantify model uncertainty within a Deep Homography Estimation framework. The method integrates with any DHE model to produce a total uncertainty $U^{\mathrm{total}}_{RS\rightarrow RT}$, enabling rejection of high-uncertainty samples and robust aggregation of displacements $\tilde{D}_{RS\rightarrow RT}$. On the Boson-nighttime dataset, UASTHN achieves strong geolocation performance and reliable uncertainty signaling, identifying failure modes across textureless, corrupted, geometric, self-similar, out-of-region, and outdated-map scenarios. The work demonstrates the practical value of combined data- and model-centered uncertainty estimation for cross-domain localization tasks, with public code and models available.

Abstract

Geo-localization is an essential component of Unmanned Aerial Vehicle (UAV) navigation systems to ensure precise absolute self-localization in outdoor environments. To address the challenges of GPS signal interruptions or low illumination, Thermal Geo-localization (TG) employs aerial thermal imagery to align with reference satellite maps to accurately determine the UAV's location. However, existing TG methods lack uncertainty measurement in their outputs, compromising system robustness in the presence of textureless or corrupted thermal images, self-similar or outdated satellite maps, geometric noises, or thermal images exceeding satellite maps. To overcome these limitations, this paper presents UASTHN, a novel approach for Uncertainty Estimation (UE) in Deep Homography Estimation (DHE) tasks for TG applications. Specifically, we introduce a novel Crop-based Test-Time Augmentation (CropTTA) strategy, which leverages the homography consensus of cropped image views to effectively measure data uncertainty. This approach is complemented by Deep Ensembles (DE) employed for model uncertainty, offering comparable performance with improved efficiency and seamless integration with any DHE model. Extensive experiments across multiple DHE models demonstrate the effectiveness and efficiency of CropTTA in TG applications. Analysis of detected failure cases underscores the improved reliability of CropTTA under challenging conditions. Finally, we demonstrate the capability of combining CropTTA and DE for a comprehensive assessment of both data and model uncertainty. Our research provides profound insights into the broader intersection of localization and uncertainty estimation. The code and models are publicly available.

Figures (6)

• Figure 1: Data Uncertainty in Thermal Geo-localization (TG): Our approach captures six categories of high data-uncertainty samples leading to TG failure, where predicted displacements significantly deviate from the ground truth. Thermal images are overlaid on predicted displacements on the satellite imagery. High-resolution images are available on our project page.
• Figure 2: UASTHN framework: CropTTA augments thermal images, and network $F_H$ with an UE module calculates aggregated displacements ($\tilde{D}_{RS\rightarrow RT}$) and data uncertainty ($U^\textrm{TTA}_{RS\rightarrow RT}$). $U^\textrm{TTA}_{RS\rightarrow RT}$ is used to reject samples with high uncertainty. Optionally, DE estimates model uncertainty ($U^\textrm{DE}_{RS\rightarrow RT}$), which can be combined with CropTTA for comprehensive UE.
• Figure 3: Ablation Study. We use success rate and Validation (Val) MACE metrics to ablate the training and evaluation settings. Baseline indicates STHN STHN two-stage baseline performance.
• Figure 4: ROC curves for CropTTA with STHN two-stage methods STHN across different $D_C$, with predictions exceeding $25~m$ MACE considered as expected rejected predictions.
• Figure 5: MACE histogram for CropTTA with STHN two-stage methods across different $D_C$