DC-VLAQ: Query-Residual Aggregation for Robust Visual Place Recognition

TL;DR

DC-VLAQ addresses robust visual place recognition under large appearance and domain shifts by coupling residual-guided fusion of multiple visual foundation models with a stable, query-based global descriptor. It anchors fusion in the DINOv2 space while injecting CLIP semantics through a learnable residual, producing fused local tokens $Z_i = X_i^D + F_C(X_i^C - X_i^D)$ that preserve retrieval geometry. The global descriptor is built with Vector of Local Aggregated Queries (VLAQ), a residual-based, query-centered aggregation using $B$ blocks of $S$ learnable queries to encode local deviations $v_{ik}$ around prototypes, yielding robust representations. Across Pitts30k, Tokyo24/7, MSLS, Nordland, SPED, and AmsterTime, DC-VLAQ achieves state-of-the-art or highly competitive performance, especially under long-term and cross-domain changes, validating the stability and discriminability benefits of the residual fusion and residual query-based aggregation.

Abstract

One of the central challenges in visual place recognition (VPR) is learning a robust global representation that remains discriminative under large viewpoint changes, illumination variations, and severe domain shifts. While visual foundation models (VFMs) provide strong local features, most existing methods rely on a single model, overlooking the complementary cues offered by different VFMs. However, exploiting such complementary information inevitably alters token distributions, which challenges the stability of existing query-based global aggregation schemes. To address these challenges, we propose DC-VLAQ, a representation-centric framework that integrates the fusion of complementary VFMs and robust global aggregation. Specifically, we first introduce a lightweight residual-guided complementary fusion that anchors representations in the DINOv2 feature space while injecting complementary semantics from CLIP through a learned residual correction. In addition, we propose the Vector of Local Aggregated Queries (VLAQ), a query--residual global aggregation scheme that encodes local tokens by their residual responses to learnable queries, resulting in improved stability and the preservation of fine-grained discriminative cues. Extensive experiments on standard VPR benchmarks, including Pitts30k, Tokyo24/7, MSLS, Nordland, SPED, and AmsterTime, demonstrate that DC-VLAQ consistently outperforms strong baselines and achieves state-of-the-art performance, particularly under challenging domain shifts and long-term appearance changes.

Figures (4)

• Figure 1: Comparison of VPR pipelines: (Top) Most existing VPR methods rely on a single VFM to extract local tokens, which ignores complementary cues across different models. (Mid.) Naive fusion of heterogeneous VFM tokens results in degraded performance, due to distorted distributions and unstable retrieval geometry. (Bot.) Our DC-VLAQ introduces Residual-Guided Complementary Fusion to preserve the original token distribution while injecting complementary information, and Query--Residual Global Aggregation to achieve stable and discriminative global descriptors.
• Figure 2: Overview of the proposed DC-VLAQ pipeline. An input image is first encoded by DINOv2 and CLIP to extract complementary local features. Then, a residual-guided fusion module injects semantic information from CLIP as residual corrections anchored to DINOv2 features. Finally, the fused tokens are aggregated by the proposed VLAQ aggregator to produce a compact global descriptor for nearest-neighbor place retrieval.
• Figure 3: Qualitative comparison of retrieval results on challenging VPR benchmarks. The first four rows show results on AmsterTime, characterized by severe historical domain shifts across years, while the last row reports results on SPED, featuring cross-domain surveillance imagery and extreme appearance changes. For each query (leftmost), we visualize the top-1 retrieved images produced by different methods, with correct retrievals highlighted in green and incorrect ones in red.
• Figure 4: Visualization of query activation heatmaps. The first five rows correspond to MSLS-val, and the last row shows examples from AmsterTime. For each query (a), we visualize the activation heatmaps produced by pre-trained DINOv2 (b), pre-trained CLIP (c), and our DC-VLAQ (d). Observe the complementarity of DINOv2 and CLIP, and the adaptation of our DC-VLAQ to the VPR task, e.g., by suppressing non-discriminative road features.