Joint Multi-Condition Representation Modelling via Matrix Factorisation for Visual Place Recognition

TL;DR

This paper tackles multi-reference visual place recognition under appearance and viewpoint changes by proposing a training-free, descriptor-level mapping that jointly models multiple reference observations. Each place is represented as a subspace formed from its multi-condition descriptors via QR decomposition, enabling projection-based residual matching without retraining. Empirical results show recall gains up to about 18% on multi-appearance data, up to about 10% on multi-viewpoint data, and around 5% on unstructured references, while maintaining low computational overhead and descriptor-backbone agnosticism. The SotonMV benchmark further enables systematic evaluation of multi-view representations, highlighting the method's practical relevance for robust, edge-friendly localisation in robotics and autonomous systems.

Abstract

We address multi-reference visual place recognition (VPR), where reference sets captured under varying conditions are used to improve localisation performance. While deep learning with large-scale training improves robustness, increasing data diversity and model complexity incur extensive computational cost during training and deployment. Descriptor-level fusion via voting or aggregation avoids training, but often targets multi-sensor setups or relies on heuristics with limited gains under appearance and viewpoint change. We propose a training-free, descriptor-agnostic approach that jointly models places using multiple reference descriptors via matrix decomposition into basis representations, enabling projection-based residual matching. We also introduce SotonMV, a structured benchmark for multi-viewpoint VPR. On multi-appearance data, our method improves Recall@1 by up to ~18% over single-reference and outperforms multi-reference baselines across appearance and viewpoint changes, with gains of ~5% on unstructured data, demonstrating strong generalisation while remaining lightweight.

Figures (7)

• Figure 1: Top: Each place is represented by multiple images with different conditions, whose descriptors are stacked and subspace is formed using matrix factorisation, here QR decomposition, to construct the map. Bottom: At inference, the query is projected and reconstructed using each subspace, with the smallest residual determining the match.
• Figure 2: Example places from datasets we evaluate. These datasets cover multi-reference sets with varying viewpoints, illumination, seasons and weather. Notably, the GLDv2-micro dataset contains unstructured images of varying conditions, size and channel compositions.
• Figure 3: (Left) The area considered in Southampton. Each 'point' is a cluster of scenes, with each scene encompassing four images at regularly spaced orientation. (Right) Example places from SotonMV. Each place is traversed multiple times (appearance change), and captured at four headings (viewpoint change). The query images are taken at intermediate headings, representing VPR under challenging viewpoint with non-trivial spatial overlap.
• Figure 4: (Left, Middle) Distribution of scenes in SotonMV over years and months. (Right) Number of unique dates per place. Each place is composed of multiple scenes at different dates, where each scene consists of 4 images at different orientations.
• Figure 5: Recall@1 versus descriptor size. Our QR-based approach using smaller descriptor sizes often outperforms other multi-reference approaches at maximum size, offering optimization with map memory. Note: for each plot the y-axis is scaled respectively for clear visualisation.