Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Invariance on Manifolds: Understanding Robust Visual Representations for Place Recognition

Jintao Cheng, Weibin Li, Zhijian He, Jin Wu, Chi Man Vong, Wei Zhang

TL;DR

The paper tackles Visual Place Recognition under drastic appearance and viewpoint changes by proposing a training-free framework that models scenes as second-order covariance descriptors on the SPD manifold $\mathcal{S}_{++}^d$. It introduces the Riemannian Invariant Aggregation (RIA) pipeline, which projects local features, denoises their covariance with ReCov, and maps the SPD descriptor to a linear tangent space via the Power Euclidean Metric, followed by isometric vectorization for Euclidean retrieval. The approach yields strong zero-shot generalization, achieves competitive or superior performance against supervised baselines on several benchmarks, and demonstrates robustness to illumination and viewpoint perturbations with theoretical guarantees and empirical validation. This work provides a principled, geometry-first alternative to learnable aggregation, enabling robust, training-free VPR suitable for open-world deployment.

Abstract

Visual Place Recognition (VPR) demands representations robust to drastic environmental and viewpoint shifts. Current aggregation paradigms, however, either rely on data-hungry supervision or simplistic first-order statistics, often neglecting intrinsic structural correlations. In this work, we propose a Second-Order Geometric Statistics framework that inherently captures geometric stability without training. We conceptualize scenes as covariance descriptors on the Symmetric Positive Definite (SPD) manifold, where perturbations manifest as tractable congruence transformations. By leveraging geometry-aware Riemannian mappings, we project these descriptors into a linearized Euclidean embedding, effectively decoupling signal structure from noise. Our approach introduces a training-free framework built upon fixed, pre-trained backbones, achieving strong zero-shot generalization without parameter updates. Extensive experiments confirm that our method achieves highly competitive performance against state-of-the-art baselines, particularly excelling in challenging zero-shot scenarios.

Invariance on Manifolds: Understanding Robust Visual Representations for Place Recognition

TL;DR

The paper tackles Visual Place Recognition under drastic appearance and viewpoint changes by proposing a training-free framework that models scenes as second-order covariance descriptors on the SPD manifold . It introduces the Riemannian Invariant Aggregation (RIA) pipeline, which projects local features, denoises their covariance with ReCov, and maps the SPD descriptor to a linear tangent space via the Power Euclidean Metric, followed by isometric vectorization for Euclidean retrieval. The approach yields strong zero-shot generalization, achieves competitive or superior performance against supervised baselines on several benchmarks, and demonstrates robustness to illumination and viewpoint perturbations with theoretical guarantees and empirical validation. This work provides a principled, geometry-first alternative to learnable aggregation, enabling robust, training-free VPR suitable for open-world deployment.

Abstract

Visual Place Recognition (VPR) demands representations robust to drastic environmental and viewpoint shifts. Current aggregation paradigms, however, either rely on data-hungry supervision or simplistic first-order statistics, often neglecting intrinsic structural correlations. In this work, we propose a Second-Order Geometric Statistics framework that inherently captures geometric stability without training. We conceptualize scenes as covariance descriptors on the Symmetric Positive Definite (SPD) manifold, where perturbations manifest as tractable congruence transformations. By leveraging geometry-aware Riemannian mappings, we project these descriptors into a linearized Euclidean embedding, effectively decoupling signal structure from noise. Our approach introduces a training-free framework built upon fixed, pre-trained backbones, achieving strong zero-shot generalization without parameter updates. Extensive experiments confirm that our method achieves highly competitive performance against state-of-the-art baselines, particularly excelling in challenging zero-shot scenarios.
Paper Structure (27 sections, 4 theorems, 20 equations, 5 figures, 6 tables)

This paper contains 27 sections, 4 theorems, 20 equations, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Visual Place Recognition
  4. Deep Learning on SPD Manifolds
  5. Preliminaries
  6. Methodology
  7. Feature Projection and Aggregation
  8. Sparse Structural Rectification
  9. Iterative Riemannian Linearization
  10. Isometric Vectorization
  11. Experiments
  12. Compared Baselines
  13. Implementation Details
  14. Datasets
  15. Main Results
  16. ...and 12 more sections

Key Result

Theorem B.2

For any $\boldsymbol{C}_1, \boldsymbol{C}_2 \in \mathcal{S}_{++}^d$ and any orthogonal matrix $\boldsymbol{Q} \in O(d)$, the PEM distance with $\alpha=0.5$ is invariant:

Figures (5)

  • Figure 1: Illustration of the two fundamental challenges in Visual Place Recognition (VPR). (a) Condition Invariance: The system must identify the correct match (right) despite drastic illumination and weather changes (e.g., sunny day vs. rainy night), while rejecting perceptually similar scenes from different locations (middle). (b) Viewpoint Invariance: The system must recognize the same landmark (right) under significant changes in scale and camera pose, distinguishing it from other similar-looking structures (middle).
  • Figure 2: Schematic overview of the proposed Riemannian Invariant Aggregation (RIA) framework. The pipeline transforms local features from a frozen backbone into a robust global descriptor through four geometric phases: Stage 1: High-dimensional features are projected onto a lower-dimensional subspace to ensure a full-rank covariance estimation. Stage 2: We compute the sample covariance and apply ReCov to suppress spurious noise. Stage 3: The covariance descriptor on the SPD manifold is mapped to a linearized tangent space via the PEM, approximated by Newton-Schulz iterations. Stage 4: The matrix is flattened using isometric vectorization (scaling off-diagonals by $\sqrt{2}$) and $L_2$ normalized to produce the final retrieval-ready descriptor. The entire process is training-free and parameter-efficient.
  • Figure 3: Feature distance drift under illumination and viewpoint changes.
  • Figure 4: Qualitative comparison of Top-3 retrieval results. Comparison between our RIA (top row) and the VLAD baseline (bottom row). (a) Under drastic illumination changes (Night-to-Day on Tokyo24/7), RIA successfully retrieves the correct scene. (b) Under viewpoint and scale variations (Pitts30k), RIA accurately identifies the landmark. Green and red borders indicate correct and incorrect matches, respectively.
  • Figure 5: Qualitative retrieval results under challenging conditions. Each row represents a different challenge scenario: (1) Seasonal Variation, (2) Occlusion, (3) Illumination Change, and (4) Perspective Change. For each scenario, we show the query image (blue border), the ground truth match, and the top-3 retrieved results. Green borders indicate correct matches, while red borders denote incorrect retrievals. Our method demonstrates strong robustness across diverse environmental perturbations, successfully retrieving correct matches even under significant appearance variations.

Theorems & Definitions (11)

  • Theorem B.2
  • proof
  • Remark B.3
  • Theorem C.2
  • proof
  • Remark C.3
  • Theorem D.1
  • proof
  • Remark D.2
  • Theorem E.1
  • ...and 1 more