Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random matrix theory improved Fréchet mean of symmetric positive definite matrices

Florent Bouchard, Ammar Mian, Malik Tiomoko, Guillaume Ginolhac, Frédéric Pascal

TL;DR

The paper tackles the problem of computing Fréchet means of symmetric positive definite covariance matrices in high-dimensional, low-sample regimes. It introduces an RMT-based correction to the squared Fisher distance and a one-step Fréchet mean algorithm that operates directly on the SPD manifold, leveraging random-matrix theory to reduce dependence on limited data. It also provides an improved covariance estimator and extends the RMT mean to learning tasks like Nearest Centroid and K-Means, demonstrating advantages on synthetic data and real EEG and hyperspectral datasets. The findings indicate improved robustness and accuracy when averaging many covariance matrices under modest sample sizes, offering practical impact for covariance-based learning in challenging data regimes.

Abstract

In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fréchet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine-learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory-based method that estimates Fréchet means, which is particularly beneficial when dealing with low sample support and a high number of matrices to average. Our experimental evaluation, involving both synthetic and real-world EEG and hyperspectral datasets, shows that we largely outperform state-of-the-art methods.

Random matrix theory improved Fréchet mean of symmetric positive definite matrices

TL;DR

The paper tackles the problem of computing Fréchet means of symmetric positive definite covariance matrices in high-dimensional, low-sample regimes. It introduces an RMT-based correction to the squared Fisher distance and a one-step Fréchet mean algorithm that operates directly on the SPD manifold, leveraging random-matrix theory to reduce dependence on limited data. It also provides an improved covariance estimator and extends the RMT mean to learning tasks like Nearest Centroid and K-Means, demonstrating advantages on synthetic data and real EEG and hyperspectral datasets. The findings indicate improved robustness and accuracy when averaging many covariance matrices under modest sample sizes, offering practical impact for covariance-based learning in challenging data regimes.

Abstract

In this study, we consider the realm of covariance matrices in machine learning, particularly focusing on computing Fréchet means on the manifold of symmetric positive definite matrices, commonly referred to as Karcher or geometric means. Such means are leveraged in numerous machine-learning tasks. Relying on advanced statistical tools, we introduce a random matrix theory-based method that estimates Fréchet means, which is particularly beneficial when dealing with low sample support and a high number of matrices to average. Our experimental evaluation, involving both synthetic and real-world EEG and hyperspectral datasets, shows that we largely outperform state-of-the-art methods.
Paper Structure (23 sections, 3 theorems, 30 equations, 2 figures, 2 tables, 4 algorithms)

This paper contains 23 sections, 3 theorems, 30 equations, 2 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Random matrix theory
  4. Covariance estimation
  5. Distance estimation
  6. Riemannian optimization on $\mathcal{S}^{++}_{p}$
  7. Closely related work
  8. RMT improved covariance estimation
  9. Covariance estimator algorithm
  10. Simulations summary and concluding remarks
  11. RMT corrected Fréchet mean on $\mathcal{S}^{++}_{p}$
  12. RMT mean algorithm
  13. Nearest Centroid and K-Means based on RMT
  14. Simulations
  15. Real data learning experiments
  16. ...and 8 more sections

Key Result

Theorem 2.1

Given $\boldsymbol{X}\in\mathbb{R}^{p\times n}$ ($p>n$) with SCM $\boldsymbol{\hat{C}}$ and a deterministic $\boldsymbol{R}\in\mathcal{S}^{++}_{p}$, the RMT correction of the squared Fisher distance eq:Fisher_dist is where $c=p/n<1$; $\boldsymbol{\lambda}$ and $\boldsymbol{\zeta}$ contain the eigenvalues of $\boldsymbol{R}^{-1}\boldsymbol{\hat{C}}$ and $\boldsymbol{\Lambda}-\frac{\sqrt{\boldsymbo

Figures (2)

  • Figure 1: Mean square error (MSE) over 1000 trials of the estimated Fréchet mean towards the true mean matrix with respect to the number of samples $n$ (left) and number of matrices $K$ (right). Parameters are $p=64$, $K=10$ on the left and $n=128$ on the right. Lines correspond to the medians while filled areas correspond to the $5^{\textup{th}}$ and $95^{\textup{th}}$ quantiles.
  • Figure 2: MSE of the estimated covariance. Parameters are $p=64$, $\ell_{\mathrm{max}}=100$, $\epsilon=10^{-6}$, $\alpha=10$. Plot done over 1000 trials. The line corresponds to the median and the filled area corresponds to the $5$-th and $95$-th quantiles over the trials.

Theorems & Definitions (8)

  • Theorem 2.1: RMT corrected squared Fisher distance from couillet2019random
  • Proposition 4.1
  • proof
  • Proposition 4.2
  • proof
  • Remark 4.3
  • Remark 5.1
  • Remark 5.2