Transient anisotropic kernel for probabilistic learning on manifolds

TL;DR

The paper addresses learning on stochastic manifolds from small datasets by replacing the traditional diffusion-map (DMAPS) projection basis, built from a time-independent isotropic kernel, with a transient anisotropic kernel-derived basis. This new basis is constructed from an Itô SDE whose drift aligns with the gradient of a KDE-based potential, enabling a time-evolving, data-adaptive projection that better captures statistical dependencies in heterogeneous data. The authors develop a complete theoretical framework: time-dependent kernels, corresponding operators, finite approximations, and an objective criterion based on normalized mutual information to identify the optimal transient instant; they validate the method through three applications spanning varying heterogeneity levels, showing improved concentration and joint-statistics learning. The results demonstrate that the transient anisotropic kernel can outperform the standard DMAPS-based approach for probabilistic surrogates, especially in complex, high-variance domains, while preserving asymptotic consistency with the DMAPS basis as $t o0$. This advances probabilistic learning on manifolds by enabling more accurate conditional statistics and interpretable, information-theoretically guided basis selection, with practical impact on surrogate modeling under uncertainty for computationally expensive systems.

Abstract

PLoM (Probabilistic Learning on Manifolds) is a method introduced in 2016 for handling small training datasets by projecting an Itô equation from a stochastic dissipative Hamiltonian dynamical system, acting as the MCMC generator, for which the KDE-estimated probability measure with the training dataset is the invariant measure. PLoM performs a projection on a reduced-order vector basis related to the training dataset, using the diffusion maps (DMAPS) basis constructed with a time-independent isotropic kernel. In this paper, we propose a new ISDE projection vector basis built from a transient anisotropic kernel, providing an alternative to the DMAPS basis to improve statistical surrogates for stochastic manifolds with heterogeneous data. The construction ensures that for times near the initial time, the DMAPS basis coincides with the transient basis. For larger times, the differences between the two bases are characterized by the angle of their spanned vector subspaces. The optimal instant yielding the optimal transient basis is determined using an estimation of mutual information from Information Theory, which is normalized by the entropy estimation to account for the effects of the number of realizations used in the estimations. Consequently, this new vector basis better represents statistical dependencies in the learned probability measure for any dimension. Three applications with varying levels of statistical complexity and data heterogeneity validate the proposed theory, showing that the transient anisotropic kernel improves the learned probability measure.

Figures (14)

• Figure 1: Criteria defined in Lemma \ref{['lemma:5']} for controlling the convergence of the Gaussian-case reference with $\nu=1$, $n_s=1$, $n_d=1200$, and $n_{\rm{MC}} = 1200$.
• Figure 2: (a) Comparison of the reference eigenvalues $\lambda_{r,\alpha}$ (diamond) with the computed eigenvalues ${ \hat{ \lambda }}_\alpha$ (circle) for $\alpha=0,1, \ldots,5$. (b) Graph of $n_d\mapsto \hbox{err}_{\lambda}(n_d)$ quantifying the relative error between $\alpha\mapsto\lambda_{r,\alpha}$ and $\alpha\mapsto{ \hat{ \lambda }}_\alpha$.
• Figure 3: Application 1. Probability density function (pdf) of components $1$, $2$, $4$, and $5$ for ${\bm{H}}$ estimated with the $n_d$ realizations of the training dataset (thin black line) and pdf estimated with $n_{\hbox{ar}}$ learned realizations (thick blue line), for ${\bm{H}}_{\hbox{ar}}$ using MCMC without PLoM (a,d,g,j), for ${\bm{H}}_{\hbox{DB}}$ using PLoM with RODB (b,e,h,k), and for ${\bm{H}}_{\hbox{TB}}$ using PLoM with ROTB$(n_{\hbox{opt}} \Delta t)$ (c, f, i, l).
• Figure 4: Application 1. Joint probability density function of components $4$ with $5$ of ${\bm{H}}$ estimated with the $n_d$ realizations of the training dataset (a) and estimated with $n_{\hbox{ar}}$ learned realizations, for ${\bm{H}}_{\hbox{ar}}$ using MCMC without PLoM (b), for ${\bm{H}}_{\hbox{DB}}$ using PLoM with RODB (c), and for ${\bm{H}}_{\hbox{TB}}$ using PLoM with ROTB$(n_{\hbox{opt}} \Delta t)$ (d).
• Figure 5: Application 1. Clouds of $n_{\hbox{ar}}$ points corresponding to $n_{\hbox{ar}}$ learned realizations, for components $1$, $2$, $3$ (a,b,c) and components $3$, $4$, $5$ (d,e,f), for ${\bm{H}}_{\hbox{ar}}$ using MCMC without PLoM (a,d), for ${\bm{H}}_{\hbox{DB}}$ using PLoM with RODB (b,e), and for ${\bm{H}}_{\hbox{TB}}$ using PLoM with ROTB$(n_{\hbox{opt}} \Delta t)$ (c,f).

Theorems & Definitions (33)

• remark 1: Another algebraic representation of operator $\hat{L}_{\hbox{FKP}}$
• definition 1: Kernel $k_t$ on ${\mathbb{R}}^\nu\times{\mathbb{R}}^\nu$
• Lemma 1: Properties of kernel $k_t$
• proof
• definition 2: Hilbert space ${\mathbb{H}}= L^2({\mathbb{R}}^\nu;p_{\bm{H}})$
• Lemma 2: Hilbert basis in ${\mathbb{H}}$
• proof
• Proposition 1: Spectral representation of kernel $k_t$
• proof
• definition 3: Operator $K_t$ associated with kernel $k_t$