Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Cross-Device Localization via Neural Metric Learning and Feature Fusion

Meixia Lin, Mingkai Liu, Shuxue Peng, Dikai Fan, Shengyu Gu, Xianliang Huang, Haoyang Ye, Xiao Liu

TL;DR

The paper tackles cross-device camera localization where traditional geometry can struggle under calibration and viewpoint differences. It introduces a hybrid pipeline with a shared retrieval encoder and dual branches: a geometric PnP-based path and a neural metric path using MapAnything, augmented by neural-guided candidate pruning and depth conditioning. Key contributions include a multi-descriptor feature fusion retrieval, a robust hybrid pose estimation strategy, neural pruning for candidate filtering, and depth-conditioned metric localization. Empirical results on HYDRO and SUCCU show a final score of 92.62 (R@0.5m, 5°), indicating strong robustness and precision across devices and scenes.

Abstract

We present a hybrid cross-device localization pipeline developed for the CroCoDL 2025 Challenge. Our approach integrates a shared retrieval encoder and two complementary localization branches: a classical geometric branch using feature fusion and PnP, and a neural feed-forward branch (MapAnything) for metric localization conditioned on geometric inputs. A neural-guided candidate pruning strategy further filters unreliable map frames based on translation consistency, while depth-conditioned localization refines metric scale and translation precision on Spot scenes. These components jointly lead to significant improvements in recall and accuracy across both HYDRO and SUCCU benchmarks. Our method achieved a final score of 92.62 (R@0.5m, 5°) during the challenge.

Hybrid Cross-Device Localization via Neural Metric Learning and Feature Fusion

TL;DR

The paper tackles cross-device camera localization where traditional geometry can struggle under calibration and viewpoint differences. It introduces a hybrid pipeline with a shared retrieval encoder and dual branches: a geometric PnP-based path and a neural metric path using MapAnything, augmented by neural-guided candidate pruning and depth conditioning. Key contributions include a multi-descriptor feature fusion retrieval, a robust hybrid pose estimation strategy, neural pruning for candidate filtering, and depth-conditioned metric localization. Empirical results on HYDRO and SUCCU show a final score of 92.62 (R@0.5m, 5°), indicating strong robustness and precision across devices and scenes.

Abstract

We present a hybrid cross-device localization pipeline developed for the CroCoDL 2025 Challenge. Our approach integrates a shared retrieval encoder and two complementary localization branches: a classical geometric branch using feature fusion and PnP, and a neural feed-forward branch (MapAnything) for metric localization conditioned on geometric inputs. A neural-guided candidate pruning strategy further filters unreliable map frames based on translation consistency, while depth-conditioned localization refines metric scale and translation precision on Spot scenes. These components jointly lead to significant improvements in recall and accuracy across both HYDRO and SUCCU benchmarks. Our method achieved a final score of 92.62 (R@0.5m, 5°) during the challenge.
Paper Structure (8 sections, 1 figure, 3 tables)

This paper contains 8 sections, 1 figure, 3 tables.

Table of Contents

  1. Method Overview
  2. Unified Retrieval Encoder and Dual Localization Branches
  3. Feature Fusion Retrieval Pipeline
  4. Hybrid Pose Estimation (PnP + MapAnything)
  5. Neural-guided Candidate Pruning
  6. Depth Conditioning and Filtering
  7. Results and Discussion
  8. Conclusion

Figures (1)

  • Figure 1: Hybrid localization pipeline. Our system consists of two stages: mapping (top) and localization (bottom). During mapping, input images are processed by keypoint detection (SuperPoint, GIM-finetuned SuperPoint, DISK) and feature matching modules to construct the 3D map via triangulation. In the localization stage, MegaLoc retrieval selects the top-100 candidate map frames for each query from the constructed map. The retrieved pairs are then processed by the classical branch (feature matching + RANSAC-PnP) and the neural branch (MapAnything). A neural-guided map filtering step prunes candidate frames with large translation distances before re-localization. Finally, a hybrid 6-DoF pose is produced by fusing PnP and neural predictions, achieving robust and accurate cross-device localization.