Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Categorical and geometric methods in statistical, manifold, and machine learning

Hông Vân Lê, Hà Quang Minh, Frederic Protin, Wilderich Tuschmann

TL;DR

This work advances a categorical and geometric perspective on statistical and machine learning by developing the category of probabilistic morphisms as a natural language for Markov kernels and regular conditional probabilities, and by integrating generative supervised-learning models with RKHS-based correct losses. It demonstrates learnability results for overparameterized models through a Vapnik-Stepanuk-inspired perturbation framework, and extends kernel methods to non-Euclidean geometries, including Log-Euclidean and Log-Hilbert-Schmidt metrics, with practical implications via RKHS covariance operators and Gram-based computations. The geometric kernel section shows when Gaussian-type kernels remain PD on SPD manifolds and how Bregman divergences (notably Alpha Log-Det) yield PD kernels, both in finite and infinite dimensions. On the manifold-learning side, the paper surveys Belkin-Niyogi Laplacian-based spectral methods and Fefferman et al.’s Riemannian reconstruction, illustrating how data-driven approaches can recover intrinsic geometric structure and enable differential-geometric analysis. Together, these contributions provide a cohesive framework for probabilistic, geometric, and spectral tools in statistical learning with potential impact on theory and data-driven applications.

Abstract

We present and discuss applications of the category of probabilistic morphisms, initially developed in \cite{Le2023}, as well as some geometric methods to several classes of problems in statistical, machine and manifold learning which shall be, along with many other topics, considered in depth in the forthcoming book \cite{LMPT2024}.

Categorical and geometric methods in statistical, manifold, and machine learning

TL;DR

This work advances a categorical and geometric perspective on statistical and machine learning by developing the category of probabilistic morphisms as a natural language for Markov kernels and regular conditional probabilities, and by integrating generative supervised-learning models with RKHS-based correct losses. It demonstrates learnability results for overparameterized models through a Vapnik-Stepanuk-inspired perturbation framework, and extends kernel methods to non-Euclidean geometries, including Log-Euclidean and Log-Hilbert-Schmidt metrics, with practical implications via RKHS covariance operators and Gram-based computations. The geometric kernel section shows when Gaussian-type kernels remain PD on SPD manifolds and how Bregman divergences (notably Alpha Log-Det) yield PD kernels, both in finite and infinite dimensions. On the manifold-learning side, the paper surveys Belkin-Niyogi Laplacian-based spectral methods and Fefferman et al.’s Riemannian reconstruction, illustrating how data-driven approaches can recover intrinsic geometric structure and enable differential-geometric analysis. Together, these contributions provide a cohesive framework for probabilistic, geometric, and spectral tools in statistical learning with potential impact on theory and data-driven applications.

Abstract

We present and discuss applications of the category of probabilistic morphisms, initially developed in \cite{Le2023}, as well as some geometric methods to several classes of problems in statistical, machine and manifold learning which shall be, along with many other topics, considered in depth in the forthcoming book \cite{LMPT2024}.
Paper Structure (21 sections, 15 theorems, 119 equations)

This paper contains 21 sections, 15 theorems, 119 equations.

Table of Contents

  1. Introduction
  2. Probabilistic morphisms and generative models of supervised learning
  3. Notation and conventions
  4. The category $\mathbf{Probm}\,$ and characterizations of regular conditional probability measures
  5. Category of probabilistic morphisms
  6. Characterizations of regular conditional probability measures
  7. Generative models of supervised learning
  8. Generative models of supervised learning and correct loss functions
  9. RKHSs and correct loss functions
  10. Leanability of overparameterized supervised learning models
  11. Learnability of a supervised learning model and regularized ERM algorithms
  12. Examples of overparameterized supervised learning models
  13. Geometric kernels and applications
  14. Kernels defined via Riemannian metrics
  15. Kernels defined with Bregman divergences
  16. ...and 6 more sections

Key Result

Proposition 2.1

Chentsov1972 Let $S$ assign to each measurable space ${\mathcal{X}}$ the Banach space ${\mathcal{S}}({\mathcal{X}})$ of finite signed measures on ${\mathcal{X}}$ endowed with the total variation norm $\| \cdot \|_{TV}$ and to every Markov kernel $T_{{\mathcal{Y}}|{\mathcal{X}}}$ the Markov homomorph

Theorems & Definitions (32)

  • Proposition 2.1
  • Remark 2.2
  • Definition 2.3
  • Remark 2.4
  • Definition 2.5: Almost surely equality
  • Theorem 2.6: Characterization of regular conditional probability measures
  • Example 2.7: Measurement error
  • Definition 2.8
  • Example 2.9
  • Remark 2.10
  • ...and 22 more