MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering

TL;DR

MutualVPR tackles supervision inconsistency in Visual Place Recognition caused by viewpoint changes and occlusions by coupling unsupervised view self-classification with descriptor learning in a mutual learning loop. It initializes coarse place labels from geo grids and iteratively refines them within each grid using K-means guided by Large Margin Cosine Loss, while the feature encoder (DINOv2 with MulConv adapter) and clustering co-evolve to align supervision with visual semantics. The approach achieves competitive to state-of-the-art results across standard VPR benchmarks and demonstrates strong robustness to occlusion, without relying on orientation labels. Practical impact includes improved long-term localization and loop-closure reliability in real-world, cluttered environments, with plans to extend to dynamic clustering strategies in future work.

Abstract

Visual Place Recognition (VPR) enables robust localization through image retrieval based on learned descriptors. However, drastic appearance variations of images at the same place caused by viewpoint changes can lead to inconsistent supervision signals, thereby degrading descriptor learning. Existing methods either rely on manually defined cropping rules or labeled data for view differentiation, but they suffer from two major limitations: (1) reliance on labels or handcrafted rules restricts generalization capability; (2) even within the same view direction, occlusions can introduce feature ambiguity. To address these issues, we propose MutualVPR, a mutual learning framework that integrates unsupervised view self-classification and descriptor learning. We first group images by geographic coordinates, then iteratively refine the clusters using K-means to dynamically assign place categories without orientation labels. Specifically, we adopt a DINOv2-based encoder to initialize the clustering. During training, the encoder and clustering co-evolve, progressively separating drastic appearance variations of the same place and enabling consistent supervision. Furthermore, we find that capturing fine-grained image differences at a place enhances robustness. Experiments demonstrate that MutualVPR achieves state-of-the-art (SOTA) performance across multiple datasets, validating the effectiveness of our framework in improving view direction generalization, occlusion robustness.

Figures (6)

• Figure 1: The problem of inconsistent supervision in existing VPR researches. The proposed mutual-learning framework define place labels by adaptive clustering in embedding space, enforcing supervision consistency.
• Figure 2: Supervision Inconsistency in Classification-based Methods. The left panel shows t-SNE visualizations of image descriptors extracted from a single geo-grid using different methods on the SF-XL dataset. “orientation labels” indicate samples colored according to their assigned view directions, while “clustering labels” refers to labels obtained by applying K-means clustering to the descriptors. The top-right panel illustrates issues related to view variation, using image examples drawn from samples in the t-SNE visualization on the left. The bottom-right panel illustrates occlusion induced issues, using image example from the same reference point.
• Figure 3: Mutual Learning Framework via Adaptive Clustering. We initialize spatial grids using UTM coordinates and assign coarse intra-grid categories. Features are extracted using DINOv2oquab2023dinov2 with adapter and GeM radenovic2018fine pooling, while adaptive clustering where iterative K-means is guided by LMCL loss dynamically refines view direction categories within grids. Clusters and features co-evolve: updated clusters supervise feature learning (stage 1), and improved features guide reclustering (stage 2), enabling robust supervision under occlusions and view direction changes.
• Figure 4: The t-SNE visualization of clustering results on SF-XL-Occlusion before and after training. The query (black dot) is reassigned from class 1 to class 2, correcting its initial misclassification.
• Figure 5: Close-up of the query and its nearest samples. Visual inspection shows class 2 has stronger semantic similarity to the query.