Fully Geometric Panoramic Localization

TL;DR

This work tackles the privacy and scalability challenges of visual localization by proposing a fully geometric panoramic localization pipeline that operates solely on the geometry of $2D$ and $3D$ lines. It builds compact geometric representations using line distributions, principal directions $D_{2D}$ and $D_{3D}$, and distance functions $f_{2D}^{L}$, $f_{3D}^{L}$, $f_{2D}^{P}$, $f_{3D}^{P}$ to perform fast pose search, augmented by a theoretical guarantee (Theorem 1) and a translation-then-rotation refinement strategy based on line intersections. The major contributions are (i) a descriptor-free localization pipeline with efficient pose-search via pre-computed distance functions and interpolation, (ii) intersection-based refinement that aligns line-derived features on the sphere, and (iii) robust, scalable performance across large-scale maps and varying lighting, with applicability to floorplan-based localization. The results show competitive accuracy compared to descriptor-based baselines while achieving smaller map footprints and faster localization, highlighting practical potential for privacy-preserving, real-world deployment. Overall, the paper advances fully geometric localization by demonstrating effective use of line-based scene context for panoramic pose estimation. These insights pave the way for robust, privacy-conscious localization in large, dynamic environments.

Abstract

We introduce a lightweight and accurate localization method that only utilizes the geometry of 2D-3D lines. Given a pre-captured 3D map, our approach localizes a panorama image, taking advantage of the holistic 360 view. The system mitigates potential privacy breaches or domain discrepancies by avoiding trained or hand-crafted visual descriptors. However, as lines alone can be ambiguous, we express distinctive yet compact spatial contexts from relationships between lines, namely the dominant directions of parallel lines and the intersection between non-parallel lines. The resulting representations are efficient in processing time and memory compared to conventional visual descriptor-based methods. Given the groups of dominant line directions and their intersections, we accelerate the search process to test thousands of pose candidates in less than a millisecond without sacrificing accuracy. We empirically show that the proposed 2D-3D matching can localize panoramas for challenging scenes with similar structures, dramatic domain shifts or illumination changes. Our fully geometric approach does not involve extensive parameter tuning or neural network training, making it a practical algorithm that can be readily deployed in the real world. Project page including the code is available through this link: https://82magnolia.github.io/fgpl/.

Figures (8)

• Figure 1: Overview of our method. (a) We target fully geometric localization using panoramas, where we only exploit lines in 2D and 3D. (b) Our method first searches for promising poses by comparing point and line distance functions that describe the holistic distribution of lines and their intersections. Then we refine each selected pose by aligning the line intersections on the sphere.
• Figure 2: Line intersection extraction and distance function comparison. (a) We pairwise intersect lines clustered along the principal directions and obtain three groups of intersection points. (b) The intersection point clusters are used to define point distance functions. Together with line distance functions that describe the coarse scene layout, point distance functions describe the fine-grained geometry of lines which enable accurate pose search.
• Figure 3: Motivation and overview of efficient distance function comparison. (a) In large-scale localization scenarios, the rotation pool constantly changes due to the variability of principal directions in 2D and 3D. Thus exhaustively computing 3D distance functions for all possible poses on the fly leads to large runtime. (b) We instead propose to (i) decouple translation and rotation, (ii) precompute and cache 3D distance functions aligned in the canonical direction, and (iii) during localization interpolate 2D distance function values at various rotations greatly reducing computation.
• Figure 4: Pose refinement using line intersections. (a) We first match line intersections that belong to the same cluster type using mutual nearest neighbors (Cluster-Guided Matches), along with a small pair of 2D, 3D intersections that are sufficiently close together on the sphere (Close Projection Matches). The matches are then used to optimize translation, where matches are also iteratively updated similar to ICP icp. (b) Rotation is then optimized based on the final set of matches by aligning the incident line directions of cluster-guided matches. Here we exploit the fact that each cluster-guided match yields a pair of line matches.
• Figure 5: Pose error recall and runtime comparison in OmniScenes piccolo and Stanford 2D-3D-S stanford2d3d for the top-1 retrieval results. Note the runtime is plotted in log scale, and the superscript XL denotes that the baseline takes line images as input.