A Unified Hierarchical Framework for Fine-grained Cross-view Geo-localization over Large-scale Scenarios

TL;DR

UnifyGeo addresses the challenge of large-scale, fine-grained cross-view geo-localization by unifying retrieval and metric localization within a single hierarchical framework. It introduces a shared multi-granularity feature encoder, a self-attention-based feature aggregator, and a localization decoder, complemented by a re-ranking mechanism and a loss that promotes cross-task synergy. The approach yields substantial gains on the LF-CVGL task, notably achieving dramatic improvements in 1m localization recall on the VIGOR benchmark, while also delivering competitive retrieval and localization performance and improved training efficiency. This unified strategy reduces training overhead and parameter count, enabling robust, scalable geo-localization suitable for real-world large-scale scenarios.

Abstract

Cross-view geo-localization is a promising solution for large-scale localization problems, requiring the sequential execution of retrieval and metric localization tasks to achieve fine-grained predictions. However, existing methods typically focus on designing standalone models for these two tasks, resulting in inefficient collaboration and increased training overhead. In this paper, we propose UnifyGeo, a novel unified hierarchical geo-localization framework that integrates retrieval and metric localization tasks into a single network. Specifically, we first employ a unified learning strategy with shared parameters to jointly learn multi-granularity representation, facilitating mutual reinforcement between these two tasks. Subsequently, we design a re-ranking mechanism guided by a dedicated loss function, which enhances geo-localization performance by improving both retrieval accuracy and metric localization references. Extensive experiments demonstrate that UnifyGeo significantly outperforms the state-of-the-arts in both task-isolated and task-associated settings. Remarkably, on the challenging VIGOR benchmark, which supports fine-grained localization evaluation, the 1-meter-level localization recall rate improves from 1.53\% to 39.64\% and from 0.43\% to 25.58\% under same-area and cross-area evaluations, respectively. Code will be made publicly available.

Figures (5)

• Figure 1: An illustration of our hierarchical geo-localization pipeline UnifyGeo for large-scale scenarios. Given a ground-level query image and an aerial image database covering the target area, our pipeline consists of three steps: (1) retrieval, which searches the entire database for potential aerial image candidates; (2) re-ranking, which selects the aerial image most likely to cover the query location and (3) metric localization, which precisely estimates the query location using the re-ranked aerial image as reference.
• Figure 2: Overview of our proposed UnifyGeo framework. It consists of separate multi-granularity feature encoders and feature aggregators for the ground and aerial branches, as well as a localization decoder. Given a ground-level query image $I_g$ and an aerial database $\mathcal{A}$, we first perform cross-view image retrieval (Retrieval) to obtain $\mathcal{C}_k$, a set of $k$ aerial image candidates. These candidates are then re-ranked to select the top-1 matched aerial reference. Finally, cross-view metric localization (Metric Loc) is performed to produce a discrete probability distribution $D$, which is overlaid on the aerial reference to indicate the likely location of $I_g$, with deeper red denoting higher probability.
• Figure 3: Illustration of the geometric correspondence between aerial and ground views. The green triangle on the aerial image (c) corresponds to the ground-truth location of image (a), marked as GT, with a roughly matching field of view indicated by the white dashed line in (c). In contrast, the magenta square on (c) does not match (a), with its corresponding ground-view image shown in (b). Notably, the same azimuthal direction (yellow bar) in (a) and (c) displays the same scene, featuring trees and a red facade building. Conversely, the same azimuthal direction (orange bar) in (b) depicts a different scene, showing a white building.
• Figure 4: Localization accuracy at various thresholds for the LF-CVGL task on the VIGOR dataset. Our method UnifyGeo is compared to the state-of-the-art methods and combined baselines on both the same-area (a) and cross-area (b) settings.
• Figure 5: Visualization of geo-localization methods on VIGOR. The left column (a-c) presents the results of two combined baselines, while the right column (d-f) shows the results of our method. Each row corresponds to the geo-localization results for the same query-view. Each panel of (a-f) comprises: a ground-view query image (top-left), the top-3 retrieved images (bottom-left, ordered from left to right), and a localization result based on the top-1 (or re-ranked) retrieved image (right, enlarged for clarity). Retrieved images are annotated with light green or red borders to indicate matches or non-matches with the query-view. In (d-f), a "yellow pentagon" on the top-left indicates the re-ranked candidate. Localization results are marked with distinct symbols: "dark green triangle" (ground-truth), "blue cross" (Sample4Geo+CCVPE), "magenta plus" (Sample4Geo+HC-Net), and "golden star" (our method).