Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network

TL;DR

The paper tackles cross-view geolocalization by bridging street-view panoramas and satellite imagery through an Explicit Panoramic BEV Transformation (EP-BEV) and a Panorama-BEV Co-Retrieval Network that jointly leverages street-view panoramas and BEV images. This dual-branch approach preserves global layout cues from the raw panoramas while emphasizing local, near-location details via BEV representations, trained with contrastive losses and fused at inference. A new CVGlobal dataset with unfixed street-view orientations, cross-regional and cross-temporal schemes, and street-view-to-map tasks is introduced to better reflect real-world conditions. Empirical results on CVUSA, CVACT, VIGOR, and CVGlobal show state-of-the-art performance and strong generalization, underscoring the method’s practicality for robust, scalable cross-view localization in diverse environments.

Abstract

Cross-view geolocalization identifies the geographic location of street view images by matching them with a georeferenced satellite database. Significant challenges arise due to the drastic appearance and geometry differences between views. In this paper, we propose a new approach for cross-view image geo-localization, i.e., the Panorama-BEV Co-Retrieval Network. Specifically, by utilizing the ground plane assumption and geometric relations, we convert street view panorama images into the BEV view, reducing the gap between street panoramas and satellite imagery. In the existing retrieval of street view panorama images and satellite images, we introduce BEV and satellite image retrieval branches for collaborative retrieval. By retaining the original street view retrieval branch, we overcome the limited perception range issue of BEV representation. Our network enables comprehensive perception of both the global layout and local details around the street view capture locations. Additionally, we introduce CVGlobal, a global cross-view dataset that is closer to real-world scenarios. This dataset adopts a more realistic setup, with street view directions not aligned with satellite images. CVGlobal also includes cross-regional, cross-temporal, and street view to map retrieval tests, enabling a comprehensive evaluation of algorithm performance. Our method excels in multiple tests on common cross-view datasets such as CVUSA, CVACT, VIGOR, and our newly introduced CVGlobal, surpassing the current state-of-the-art approaches. The code and datasets can be found at \url{https://github.com/yejy53/EP-BEV}.

Figures (5)

• Figure 1: The goal of cross-view retrieval is to identify the georeferenced satellite image most visually similar to the street view panorama query (a). Existing methods use polar transformation to convert satellite views into Polar views and then proceed with retrieval (b). Our method employs Explicit Panoramic BEV transformation to convert street view images into the BEV perspective consistent with satellite views, while preserving the previous street view panorama to satellite retrieval path (c).
• Figure 2: The cross-view retrieval dataset CVGlobal encompasses data from various distinct style cities around the world, with red sample points representing training data and blue points indicating regional testing data (a). Since street views are captured by car-mounted cameras, they are usually centered on the road, and the north direction is not fixed (b). Additionally, CVGlobal introduces new tasks such as cross-temporal evaluation (c) and street view to map evaluation (d).
• Figure 3: Schematic of the Panorama-BEV Co-Retrieval Network. Street view images and their images transformed via EP-BEV serve as query inputs, with satellite images as reference inputs. The network comprises a street view branch focusing on matching global layout information with satellite images and a BEV branch emphasizing detailed feature matching between the nearby street view area and the satellite perspective.
• Figure 4: Schematic of the Explicit Panoramic BEV Transformation, with the upper part showing specific transformation details and the lower part displaying the results of the BEV conversion.
• Figure 5: On the VIGOR dataset, we compare our method with Samp4G's retrieval results, using blue and orange boxes to represent correct and incorrect retrievals, respectively.