HierLoc: Hyperbolic Entity Embeddings for Hierarchical Visual Geolocation

TL;DR

This work tackles the challenge of global visual geolocation by reframing it as image-to-entity alignment within a hyperbolic geographic hierarchy. It introduces HierLoc, which embeds country, region, subregion, and city entities in a Lorentz-hyperbolic space, fusing multimodal signals and refining predictions via cross-modal attention and a Geo-Weighted Hyperbolic InfoNCE loss. Key contributions include constructing a compact 240k-entity hierarchy from large-scale datasets, achieving state-of-the-art results on OSV5M with substantial improvements in fine-grained localization, and demonstrating significant inference efficiency due to sublinear search over entities. The work also shows robust performance across multiple benchmarks and backbone choices, highlighting the practical impact of geometry-aware hierarchical embeddings for scalable, interpretable global geolocation and suggesting broader applicability to other hierarchical multimodal tasks.

Abstract

Visual geolocalization, the task of predicting where an image was taken, remains challenging due to global scale, visual ambiguity, and the inherently hierarchical structure of geography. Existing paradigms rely on either large-scale retrieval, which requires storing a large number of image embeddings, grid-based classifiers that ignore geographic continuity, or generative models that diffuse over space but struggle with fine detail. We introduce an entity-centric formulation of geolocation that replaces image-to-image retrieval with a compact hierarchy of geographic entities embedded in Hyperbolic space. Images are aligned directly to country, region, subregion, and city entities through Geo-Weighted Hyperbolic contrastive learning by directly incorporating haversine distance into the contrastive objective. This hierarchical design enables interpretable predictions and efficient inference with 240k entity embeddings instead of over 5 million image embeddings on the OSV5M benchmark, on which our method establishes a new state-of-the-art performance. Compared to the current methods in the literature, it reduces mean geodesic error by 19.5\%, while improving the fine-grained subregion accuracy by 43%. These results demonstrate that geometry-aware hierarchical embeddings provide a scalable and conceptually new alternative for global image geolocation.

Figures (7)

• Figure 1: HierLoc. Overall architecture. Images are encoded and mapped with $\exp_0$ into the Lorentz model of Hyperbolic space, while entities (countries, regions, subregions, cities) combine image, text, and location features. In the tangent space at the origin, cross-modal attention aligns each image with entities per hierarchy level; the resulting attention outputs are fused and projected back via $\exp_O$. Entity embeddings are not updated with cross-attention context, while image embeddings are updated using the context of cross-modal attention. Training employs our proposed Geo-Weighted Hyperbolic InfoNCE (GWH-InfoNCE), which reweights negatives with the haversine formula between image and negative entity coordinates.
• Figure 2: Illustration of $\exp_{O}$ and $\log_O$ projection functions projecting points from Tangent space to Hyperbolic space and vice versa.
• Figure 3: Comparison of accuracy and search time tradeoff with different beam widths.
• Figure 4: Comparison of computational efficiency across methods. We report wall-clock search time, throughput (queries per second), storage footprint, and FLOPs per query. Arrows indicate whether lower or higher values are better.
• Figure 5: Mean geographic error distribution of HierLoc on the OSV5M dataset.