Query-Based Adaptive Aggregation for Multi-Dataset Joint Training Toward Universal Visual Place Recognition

TL;DR

This paper tackles universal Visual Place Recognition (VPR) by addressing dataset biases through multi-dataset joint training. It introduces Query-based Adaptive Aggregation (QAA), which uses learned query-based reference codebooks and Cross-query Similarity (CS) to expand aggregation capacity without increasing descriptor dimensionality, producing descriptors of size $C_d = C_r \times C_f$ and a similarity matrix $S = \hat{F}^\top \hat{P}$. The approach achieves balanced cross-dataset generalization and competitive peak performance versus dataset-specific models, supported by extensive ablations and qualitative visualizations. Coding-rate analysis confirms CS preserves more information than score-based methods, and results demonstrate scalable, efficient VPR suitable for large-scale deployment.

Abstract

Deep learning methods for Visual Place Recognition (VPR) have advanced significantly, largely driven by large-scale datasets. However, most existing approaches are trained on a single dataset, which can introduce dataset-specific inductive biases and limit model generalization. While multi-dataset joint training offers a promising solution for developing universal VPR models, divergences among training datasets can saturate the limited information capacity in feature aggregation layers, leading to suboptimal performance. To address these challenges, we propose Query-based Adaptive Aggregation (QAA), a novel feature aggregation technique that leverages learned queries as reference codebooks to effectively enhance information capacity without significant computational or parameter complexity. We show that computing the Cross-query Similarity (CS) between query-level image features and reference codebooks provides a simple yet effective way to generate robust descriptors. Our results demonstrate that QAA outperforms state-of-the-art models, achieving balanced generalization across diverse datasets while maintaining peak performance comparable to dataset-specific models. Ablation studies further explore QAA's mechanisms and scalability. Visualizations reveal that the learned queries exhibit diverse attention patterns across datasets. Project page: http://xjh19971.github.io/QAA.

Figures (4)

• Figure 1: Query-based Adaptive Aggregation (QAA) for Multi-Dataset Joint Training toward Universal Visual Place Recognition (VPR). QAA calculates cross-query similarity matrices between query-level image features and reference codebooks—constructed by learned queries—for output descriptors, improving the information capacity of aggregation layers and enhancing cross-domain generalization.
• Figure 2: The training framework (top) and Query-Based Adaptive Aggregation (QAA) architecture (bottom).
• Figure 3: Attention Maps Corresponding to Different Query Vectors ($Q^f_i$, $Q^f_j$, and $Q^f_k$) in $Q^f$ from The Feature Prediction Model for A Front-view Dataset (MSLS Val) and Multi-view Datasets (Pitts250k and Tokyo24/7). Each pair of images represents the same location within the same dataset but from different viewpoints.
• Figure 4: Histogram of the coding rate distributions for different aggregation paradigms.