CQVPR: Landmark-aware Contextual Queries for Visual Place Recognition

TL;DR

CQVPR addresses the challenge of visual place recognition in urban environments by integrating high-level contextual information with pixel-level features. It introduces learnable contextual queries that produce context-aware heatmaps, enabling more discriminative global and local descriptors via a Transformer-based cross-attention mechanism. A dedicated contextual query matching loss aligns representations across images from the same place, while remaining robust to different places; the approach yields state-of-the-art performance on Pitts30k, MSLS-val, Pitts250k, and SPED, with notable gains in challenging AmsterTime and SVOX settings. Overall, CQVPR bridges the gap between pixel-level and segment-level methods, improving landmark disambiguation by jointly leveraging landmarks and their surrounding context for robust VPR in real-world scenarios.

Abstract

Visual Place Recognition (VPR) aims to estimate the location of the given query image within a database of geo-tagged images. To identify the exact location in an image, detecting landmarks is crucial. However, in some scenarios, such as urban environments, there are numerous landmarks, such as various modern buildings, and the landmarks in different cities often exhibit high visual similarity. Therefore, it is essential not only to leverage the landmarks but also to consider the contextual information surrounding them, such as whether there are trees, roads, or other features around the landmarks. We propose the Contextual Query VPR (CQVPR), which integrates contextual information with detailed pixel-level visual features. By leveraging a set of learnable contextual queries, our method automatically learns the high-level contexts with respect to landmarks and their surrounding areas. Heatmaps depicting regions that each query attends to serve as context-aware features, offering cues that could enhance the understanding of each scene. We further propose a query matching loss to supervise the extraction process of contextual queries. Extensive experiments on several datasets demonstrate that the proposed method outperforms other state-of-the-art methods, especially in challenging scenarios.

Figures (7)

• Figure 1: Conceptual difference among three VPR pipelines. (a) Pixel-level methods. (b) Segment-level methods. (c) The proposed CQVPR.
• Figure 2: Comparison of cross attention maps between the previous pixel-level method SelaVPR and our CQVPR. The presented two images are from the same place. Thanks to the introduced high-level contexts, CQVPR focuses on discriminative regions (e.g., buildings and towers). While SelaVPR focuses on less informative regions.
• Figure 3: The overview of the proposed CQVPR. Learnable contextual queries are first randomly initialized and then transformed into context-aware features. The context-aware features are then fused with pixel-level features from the backbone.
• Figure 4: The illustration of the context-aware feature extraction module.
• Figure 5: Comparison of local matching between CQVPR and SelaVPR. (a) presents the local matching between images from the same place. The more matches means the better performance. (b) presents local matching between images from different places. The fewer matches the better.