BevSplat: Resolving Height Ambiguity via Feature-Based Gaussian Primitives for Weakly-Supervised Cross-View Localization

TL;DR

BevSplat addresses weakly supervised cross-view localization by representing ground-view pixels as 3D Gaussian primitives enriched with semantic features and synthesizing a BEV representation for pose estimation. The method predates IPM and heavy transformers by leveraging depth-based Gaussian splats and differentiable rendering, enabling robust handling of height ambiguity and occlusions, with an icosphere-based supervision strategy for panoramic queries. It achieves state-of-the-art or near-supervised performance on KITTI and VIGOR across pinhole and panoramic queries, demonstrating strong generalization and efficiency trade-offs. The work advances scalable, camera-alone localization suitable for GPS-denied environments, while acknowledging slower inference due to 3D Gaussian rendering and outlining avenues for faster, more compact representations.

Abstract

This paper addresses the problem of weakly supervised cross-view localization, where the goal is to estimate the pose of a ground camera relative to a satellite image with noisy ground truth annotations. A common approach to bridge the cross-view domain gap for pose estimation is Bird's-Eye View (BEV) synthesis. However, existing methods struggle with height ambiguity due to the lack of depth information in ground images and satellite height maps. Previous solutions either assume a flat ground plane or rely on complex models, such as cross-view transformers. We propose BevSplat, a novel method that resolves height ambiguity by using feature-based Gaussian primitives. Each pixel in the ground image is represented by a 3D Gaussian with semantic and spatial features, which are synthesized into a BEV feature map for relative pose estimation. Additionally, to address challenges with panoramic query images, we introduce an icosphere-based supervision strategy for the Gaussian primitives. We validate our method on the widely used KITTI and VIGOR datasets, which include both pinhole and panoramic query images. Experimental results show that BevSplat significantly improves localization accuracy over prior approaches.

Figures (9)

• Figure 1: Our BevSplat cross-view localization process begins by using a depth prediction network on a single ground-level image to acquire its depth map. This depth map is then employed for 3D reconstruction into Gaussian splats, which are finally projected to a Bird's-Eye View (BEV). In comparison to the Inverse Perspective Mapping (IPM) approach, our method demonstrates improved recovery of BEV curves, more effective handling of building occlusions, and enhanced practical localization performance.
• Figure 2: BevSplat Framework Overview. Query ground image Gaussian primitive initialization involves: (1) A pre-trained depth model for initial 3D positions ($\mu_i$). (2) A ResNet and MLP to predict offsets ($\Delta\mathbf{p}_k$), scale ($\mathbf{S}_k$), rotation ($\mathbf{R}_k$), and opacity ($O_k$). (3) A DPT-fine-tuned DINOv2 for extracting semantic features ($\mathbf{f}_i$) and confidences ($c_i$), which are then bound to these Gaussians. These feature Gaussians are subsequently rendered into BEV feature and confidence maps. Satellite image features are extracted using an identical DINOv2-DPT backbone (note: weights are shared for KITTI but differ for VIGOR, similar to G2SWeakly shi2024weakly). Localization is achieved by matching satellite features with the rendered query BEV features via cosine similarity within a sliding window.
• Figure 3: Visualization of the query ground image (up) and the estimated relative pose with respect to the satellite image (bottom right) on VIGOR dataset. The BEV image projected from the query ground image using the estimated Gaussian primitives is presented in the bottom left for each example.
• Figure 4: Visualization of query ground images (a), the corresponding BEV feature maps synthesized by IPM (b), by direct projection (c), and by the proposed BevSplat (d). The top two examples are from the KITTI dataset, while the bottom two are from the VIGOR dataset.
• Figure 5: Ablation on primitives per pixel ($N_p$) The error is minimized when $N_p=3$ on KITTI dataset.