SpatiaLoc: Leveraging Multi-Level Spatial Enhanced Descriptors for Cross-Modal Localization
Tianyi Shang, Pengjie Xu, Zhaojun Deng, Zhenyu Li, Zhicong Chen, Lijun Wu
TL;DR
SpatiaLoc tackles cross-modal localization by exploiting relative spatial relationships between objects in text descriptions and point-cloud submaps. It combines a coarse stage with a Bezier-Enhanced Object Spatial Encoder (BEOSE) for instance-level cues and a Frequency Aware Encoder (FAE) for global frequency-domain features, with a fine stage using an Uncertainty Aware Gaussian Fine Localizer (UGFL) for robust regression. Key innovations include Gaussian Aggregation (GA), a cross-modal alignment loss system that jointly optimizes $\mathcal{L}_{Global}$, $\mathcal{L}_{IS}$, and $\mathcal{L}_{IO}$, and a triple-cross-attention mechanism for global features. On KITTI360Pose, SpatiaLoc significantly outperforms state-of-the-art methods in both submap retrieval and precise localization, demonstrating stronger robustness to ambiguity and occlusion in urban-scale environments.
Abstract
Cross-modal localization using text and point clouds enables robots to localize themselves via natural language descriptions, with applications in autonomous navigation and interaction between humans and robots. In this task, objects often recur across text and point clouds, making spatial relationships the most discriminative cues for localization. Given this characteristic, we present SpatiaLoc, a framework utilizing a coarse-to-fine strategy that emphasizes spatial relationships at both the instance and global levels. In the coarse stage, we introduce a Bezier Enhanced Object Spatial Encoder (BEOSE) that models spatial relationships at the instance level using quadratic Bezier curves. Additionally, a Frequency Aware Encoder (FAE) generates spatial representations in the frequency domain at the global level. In the fine stage, an Uncertainty Aware Gaussian Fine Localizer (UGFL) regresses 2D positions by modeling predictions as Gaussian distributions with a loss function aware of uncertainty. Extensive experiments on KITTI360Pose demonstrate that SpatiaLoc significantly outperforms existing state-of-the-art (SOTA) methods.