Statistical exploration of the Manifold Hypothesis

TL;DR

This work addresses why high-dimensional data often concentrate near low-dimensional manifolds by introducing the Latent Metric Model (LMM), which generates manifold structure from latent variables via a mean correlation kernel and a Mercer feature map. It links statistical properties to geometry, proving conditions for a homeomorphism and, under stationarity, an isometry between the latent domain and the observed manifold, and shows how smooth kernels induce concentration of data onto a low-dimensional subspace. The authors then propose a practical workflow—dimension selection via Wasserstein distance, PCA embedding, spherical projection, and nearest-neighbor graph analysis—backed by a PCA-consistency theorem in HDLSS regimes, plus a spherical-extension that broadens kernel choices. They demonstrate the approach on images, single-cell transcriptomics, and temperature data, extracting interpretable latent structures (loops, trees) and testing isometry/geometry hypotheses, thereby offering a model-based justification for manifold learning and a scalable toolkit for exploratory data analysis with theoretical guarantees. Overall, the paper provides a statistical foundation for the Manifold Hypothesis and a concrete, model-guided methodology that combines linear and nonlinear dimension reduction with graph-based topology, enabling principled exploration of latent geometry in diverse data domains.

Abstract

The Manifold Hypothesis is a widely accepted tenet of Machine Learning which asserts that nominally high-dimensional data are in fact concentrated near a low-dimensional manifold, embedded in high-dimensional space. This phenomenon is observed empirically in many real world situations, has led to development of a wide range of statistical methods in the last few decades, and has been suggested as a key factor in the success of modern AI technologies. We show that rich and sometimes intricate manifold structure in data can emerge from a generic and remarkably simple statistical model -- the Latent Metric Model -- via elementary concepts such as latent variables, correlation and stationarity. This establishes a general statistical explanation for why the Manifold Hypothesis seems to hold in so many situations. Informed by the Latent Metric Model we derive procedures to discover and interpret the geometry of high-dimensional data, and explore hypotheses about the data generating mechanism. These procedures operate under minimal assumptions and make use of well known graph-analytic algorithms.

Figures (17)

• Figure 1: A collection of images reduced in dimension using PCA.
• Figure 2: Planaria example. Left: first $2$ dimensions of the PCA embedding. Right: representation of the data in $2$ dimensions obtained by first reducing to 14 dimensions using PCA, then applying $t$-SNE.
• Figure 3: Torus example. Left: grey wireframe of $\mathcal{Z}$, a torus, with colour bars indicating coordinates with respect to two circles. Both the middle and right plots show the same $n=4000$ points, $Z_1,\ldots,Z_{4000}$, which are sampled uniformly on the torus, coloured by their coordinates with respect to each of the two circles.
• Figure 4: Torus example. Both the top and bottom rows show the first 9 dimensions of $\phi(Z_i)$, $i=1,\ldots,4000$. In each row, points are coloured according to the coordinates of the underlying points $Z_{1},\ldots,Z_{n}$ with respect to the two circles shown in figure \ref{['fig:torus_Zs']}. Numerical scales are omitted to de-clutter the plots.
• Figure 5: Torus example. Blue: numerical shortest path lengths between points in $\mathcal{M}$ vs. between the corresponding points in $\mathcal{Z}$. Red: theoretical scaling relationship $\sqrt{2}$.

Theorems & Definitions (43)

• Proposition 1
• Proposition 2
• Proposition 3
• Proposition 4
• Theorem 1
• Theorem 2: Mercer's theorem
• Proposition 5
• proof
• proof : Proof of Proposition \ref{['prop:Phi_W_expansion']}
• Proposition 6