Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Log-Euclidean Metrics for SPD Matrix Learning

Ziheng Chen, Yue Song, Tianyang Xu, Zhiwu Huang, Xiao-Jun Wu, Nicu Sebe

TL;DR

The paper tackles the limitation of fixed Riemannian metrics for SPD matrix learning by introducing Adaptive Log-Euclidean Metrics (ALEMs) within a Pullback Euclidean Metrics framework. ALEMs parameterize the matrix logarithm via diagonal bases, enabling end-to-end learnability of the SPD geometry and unifying LEM and LCM as pullback constructions; the authors provide dual theory and practical differential tools for backpropagation. They demonstrate closed-form Fréchet means, multiple invariances, and effective integration into SPDNet, LieBN, RResNet, and gyro MLR, with empirical gains on HDM05, FPHA, AFEW, and NTU60 datasets. The results show that learning the metric basis enhances representational capacity with minimal computational overhead, offering a flexible and scalable approach to geometry-aware deep SPD learning with broad applicability to Riemannian blocks.

Abstract

Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity to encode underlying structural correlation in data. Many successful Riemannian metrics have been proposed to reflect the non-Euclidean geometry of SPD manifolds. However, most existing metric tensors are fixed, which might lead to sub-optimal performance for SPD matrix learning, especially for deep SPD neural networks. To remedy this limitation, we leverage the commonly encountered pullback techniques and propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM). Compared with the previous Riemannian metrics, our metrics contain learnable parameters, which can better adapt to the complex dynamics of Riemannian neural networks with minor extra computations. We also present a complete theoretical analysis to support our ALEMs, including algebraic and Riemannian properties. The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks. The efficacy of our metrics is further showcased on a set of recently developed Riemannian building blocks, including Riemannian batch normalization, Riemannian Residual blocks, and Riemannian classifiers.

Adaptive Log-Euclidean Metrics for SPD Matrix Learning

TL;DR

The paper tackles the limitation of fixed Riemannian metrics for SPD matrix learning by introducing Adaptive Log-Euclidean Metrics (ALEMs) within a Pullback Euclidean Metrics framework. ALEMs parameterize the matrix logarithm via diagonal bases, enabling end-to-end learnability of the SPD geometry and unifying LEM and LCM as pullback constructions; the authors provide dual theory and practical differential tools for backpropagation. They demonstrate closed-form Fréchet means, multiple invariances, and effective integration into SPDNet, LieBN, RResNet, and gyro MLR, with empirical gains on HDM05, FPHA, AFEW, and NTU60 datasets. The results show that learning the metric basis enhances representational capacity with minimal computational overhead, offering a flexible and scalable approach to geometry-aware deep SPD learning with broad applicability to Riemannian blocks.

Abstract

Symmetric Positive Definite (SPD) matrices have received wide attention in machine learning due to their intrinsic capacity to encode underlying structural correlation in data. Many successful Riemannian metrics have been proposed to reflect the non-Euclidean geometry of SPD manifolds. However, most existing metric tensors are fixed, which might lead to sub-optimal performance for SPD matrix learning, especially for deep SPD neural networks. To remedy this limitation, we leverage the commonly encountered pullback techniques and propose Adaptive Log-Euclidean Metrics (ALEMs), which extend the widely used Log-Euclidean Metric (LEM). Compared with the previous Riemannian metrics, our metrics contain learnable parameters, which can better adapt to the complex dynamics of Riemannian neural networks with minor extra computations. We also present a complete theoretical analysis to support our ALEMs, including algebraic and Riemannian properties. The experimental and theoretical results demonstrate the merit of the proposed metrics in improving the performance of SPD neural networks. The efficacy of our metrics is further showcased on a set of recently developed Riemannian building blocks, including Riemannian batch normalization, Riemannian Residual blocks, and Riemannian classifiers.
Paper Structure (32 sections, 18 theorems, 74 equations, 3 figures, 9 tables)

This paper contains 32 sections, 18 theorems, 74 equations, 3 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Adaptive Log-Euclidean Metrics
  4. Rethinking LEM and LCM
  5. PEMs on SPD Manifolds
  6. Adaptive Log-Euclidean Metrics
  7. Differentials of General Logarithms
  8. Properties OF ALEM
  9. Applications to SPD Neural Networks
  10. Parameters Learning
  11. Gradients Computation
  12. Parameters Updates
  13. Experiments
  14. Datasets and Settings
  15. Experimental Results
  16. ...and 17 more sections

Key Result

Theorem 3.1

$(a,b)\text{-LEM}$ is the pullback metric from the Euclidean space of $\mathcal{S}^{n}$ with an $\mathrm{O}({n})$-invariant inner product $\langle , \rangle^{(a,b)}$ by matrix logarithm. Specifically, the standard LEM is the pullback metric from the Euclidean space of $\mathcal{S}^{n}$ with the stan

Figures (3)

  • Figure 1: Accuracy Curves on the FPHA Dataset.
  • Figure 2: Visualization of Parameters in the ALog Layer on the HDM05 Dataset.
  • Figure 3: Visualization of Parameters in the ALog Layer on the FPHA Dataset.

Theorems & Definitions (52)

  • Definition 2.1: Pullback Metrics
  • Theorem 3.1
  • Corollary 3.2
  • Lemma 3.3
  • Proposition 3.4: Diffeomorphism
  • Remark 3.5
  • Theorem 3.6
  • Remark 3.7
  • Proposition 3.8: Differentials
  • Proposition 3.9: Differential as Infinite Series
  • ...and 42 more