Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distance Measure Based on an Embedding of the Manifold of K-Component Gaussian Mixture Models into the Manifold of Symmetric Positive Definite Matrices

Amit Vishwakarma, KS Subrahamanian Moosath

TL;DR

This work develops a geometry-enhanced distance for K-component Gaussian Mixture Models by embedding their statistical manifold into the symmetric positive definite (SPD) matrix manifold. It proves the embedding is an isometric submanifold, derives a pullback metric that serves as a lower bound to the Fisher-Rao distance, and obtains a practical GMM distance via the affine-invariant SPD metric. The embedded manifold has dimension $\frac{K}{2}(n+1)(n+2)-1$, and the mapping is an embedding, becoming geodesic under fixed means and uniform mixing, with equality to the Fisher-Rao metric in that regime. Empirically, the approach yields state-of-the-art texture recognition accuracy on UIUC, KTH-TIPS, and UMD datasets, outperforming KL-based divergences and DPLM-based methods while maintaining computational efficiency.

Abstract

In this paper, a distance between the Gaussian Mixture Models(GMMs) is obtained based on an embedding of the K-component Gaussian Mixture Model into the manifold of the symmetric positive definite matrices. Proof of embedding of K-component GMMs into the manifold of symmetric positive definite matrices is given and shown that it is a submanifold. Then, proved that the manifold of GMMs with the pullback of induced metric is isometric to the submanifold with the induced metric. Through this embedding we obtain a general lower bound for the Fisher-Rao metric. This lower bound is a distance measure on the manifold of GMMs and we employ it for the similarity measure of GMMs. The effectiveness of this framework is demonstrated through an experiment on standard machine learning benchmarks, achieving accuracy of 98%, 92%, and 93.33% on the UIUC, KTH-TIPS, and UMD texture recognition datasets respectively.

Distance Measure Based on an Embedding of the Manifold of K-Component Gaussian Mixture Models into the Manifold of Symmetric Positive Definite Matrices

TL;DR

This work develops a geometry-enhanced distance for K-component Gaussian Mixture Models by embedding their statistical manifold into the symmetric positive definite (SPD) matrix manifold. It proves the embedding is an isometric submanifold, derives a pullback metric that serves as a lower bound to the Fisher-Rao distance, and obtains a practical GMM distance via the affine-invariant SPD metric. The embedded manifold has dimension , and the mapping is an embedding, becoming geodesic under fixed means and uniform mixing, with equality to the Fisher-Rao metric in that regime. Empirically, the approach yields state-of-the-art texture recognition accuracy on UIUC, KTH-TIPS, and UMD datasets, outperforming KL-based divergences and DPLM-based methods while maintaining computational efficiency.

Abstract

In this paper, a distance between the Gaussian Mixture Models(GMMs) is obtained based on an embedding of the K-component Gaussian Mixture Model into the manifold of the symmetric positive definite matrices. Proof of embedding of K-component GMMs into the manifold of symmetric positive definite matrices is given and shown that it is a submanifold. Then, proved that the manifold of GMMs with the pullback of induced metric is isometric to the submanifold with the induced metric. Through this embedding we obtain a general lower bound for the Fisher-Rao metric. This lower bound is a distance measure on the manifold of GMMs and we employ it for the similarity measure of GMMs. The effectiveness of this framework is demonstrated through an experiment on standard machine learning benchmarks, achieving accuracy of 98%, 92%, and 93.33% on the UIUC, KTH-TIPS, and UMD texture recognition datasets respectively.
Paper Structure (16 sections, 8 theorems, 69 equations, 1 figure, 1 table)

This paper contains 16 sections, 8 theorems, 69 equations, 1 figure, 1 table.

Table of Contents

  1. Intordutcion
  2. Related Works
  3. Preliminaries
  4. Statistical manifold
  5. Divergence Measures
  6. Gaussian Mixture Models
  7. Manifold of Symmetric Positive Definite Matrices
  8. Embedding of the statistical manifold $\mathcal{M}$ of GMMs into the manifold $SPD_{K(n+1)}(\mathbb{R})$
  9. Result and Discussion
  10. Dataset Description and Preprocessing
  11. Feature Extraction Process
  12. GMM Parameter Estimation
  13. Embedding of GMMs
  14. Classification Framework
  15. Experimental Results
  16. ...and 1 more sections

Key Result

Theorem 2

Let $f$ be a continuous probability density function on $\mathbb{R}^d$. For any $\epsilon > 0$, there exists a Gaussian Mixture Model $p(\mathbf{x})$ with $K$ components such that .

Figures (1)

  • Figure 1: Sample data of the UIUC, KTH-TIPS, and UMD datasets.

Theorems & Definitions (10)

  • Definition 1
  • Theorem 2: Universal Approximation
  • Definition 3: Affine-Invariant Metric
  • Theorem 4
  • Theorem 5
  • Theorem 6
  • Theorem 7
  • Theorem 8: Block Matrix Inversion
  • Theorem 9: Sherman-Morrison
  • Theorem 10