Analytical Discovery of Manifold with Machine Learning

TL;DR

This work introduces GAMLA, a Global Analytical Manifold Learning framework that combines a two-round autoencoder training to yield a character map $G(\\bm{x})$ to an $m$-dimensional latent space and a complementary map $R(\\bm{x})$ to an $(n-m)$-dimensional space, producing an explicit analytic manifold equation $R(\\bm{x})=\\bm{0}$ alongside a global coordinate chart. The first round learns the manifold structure via a bottleneck of size $m$, while the second round augments the bottleneck to $n$ and trains only new weights to obtain $ ilde{\\bm{z}}=R(\\bm{x})$, enabling direct computation of differential-geometric quantities such as normals and curvature from analytic forms. The method delivers two coherent representations that decompose the latent space and capture local structure around the manifold, facilitating anomaly detection and explanation through the sign and magnitude of $\\tilde{\\bm{z}}$. Extensive experiments on synthetic manifolds, Swiss Roll, Stanford Bunny data, MALGO mouse phenotypes, and image datasets demonstrate accurate global unfolding, analytic manifold description, and interpretable anomaly categorization, with successful image interpolation in the learned character space. GAMLA thus bridges data-driven manifold learning and analytical geometry, offering a scalable, interpretable tool for exploring intrinsic data geometry and surrounding structures.

Abstract

Understanding low-dimensional structures within high-dimensional data is crucial for visualization, interpretation, and denoising in complex datasets. Despite the advancements in manifold learning techniques, key challenges-such as limited global insight and the lack of interpretable analytical descriptions-remain unresolved. In this work, we introduce a novel framework, GAMLA (Global Analytical Manifold Learning using Auto-encoding). GAMLA employs a two-round training process within an auto-encoding framework to derive both character and complementary representations for the underlying manifold. With the character representation, the manifold is represented by a parametric function which unfold the manifold to provide a global coordinate. While with the complementary representation, an approximate explicit manifold description is developed, offering a global and analytical representation of smooth manifolds underlying high-dimensional datasets. This enables the analytical derivation of geometric properties such as curvature and normal vectors. Moreover, we find the two representations together decompose the whole latent space and can thus characterize the local spatial structure surrounding the manifold, proving particularly effective in anomaly detection and categorization. Through extensive experiments on benchmark datasets and real-world applications, GAMLA demonstrates its ability to achieve computational efficiency and interpretability while providing precise geometric and structural insights. This framework bridges the gap between data-driven manifold learning and analytical geometry, presenting a versatile tool for exploring the intrinsic properties of complex data sets.

Figures (11)

• Figure 1: Illustration of GAMLA manifold learning. The training process consists of two rounds. In the first training round, the model is trained using sample points on the manifold $\mathcal{M}$. After completing this training, all weights and biases are fixed. In the second training round, only the weights and biases corresponding to the newly added nodes in the bottleneck layer are trained, using a training set comprising sample points uniformly collected from a hyperrectangle set $\mathcal{A}$ that contains the manifold $\mathcal{M}$.
• Figure 2: Estimation results of the mathematical expression for manifold with a global explicit function $x_3=-0.2x_1+0.5x_1^2+0.2x_1x_2$. (A) Scatter plot of quadric surface. (B) The red data points selected through the approximation expression and the ground truth manifold depicted in gray. (C) Comparison of the coefficients of the Taylor expansion of the estimated expressions for the quadric surface with the true expressions.
• Figure 3: Estimation results of the mathematical expression for manifolds without a global explicit function. (A) Scatter plot of three-quarter cylinder. (B) The red data points selected through the approximation expression $|R_2|<\epsilon$ and the true manifold depicted in gray. (C) The normal vectors calculated using the GAMLA analytically and using MeshLab numerically. (D) The same as (C) but with noisy sampling data cloud. (E) The Gaussian curvature (ground truth: $K_G=0$) where $x_3=0$ on the three-quarter cylinder calculated using GAMLA analytically and using CloudCompare software numerically from data cloud without noise (up) and with noise (down) respectively.
• Figure 4: Unfolding and global coordinate chart acquisition results for the Swiss Roll manifold. (A) Scatter plot of the Swiss Roll manifold. (B) The unfolding of the Swiss Roll manifold in latent space. (C) and (D) The axes in the character space corresponding to the global coordinate chart in the original space.
• Figure 5: Unfolding and global coordinate chart acquisition results for $3$D point cloud dataset. (A) Scatter plot of 3D point cloud dataset collected from the Stanford Bunny model. (B) The unfolding of the $3$D point cloud dataset in latent space. (C) and (D) illustrate that the grid lines in the character space form quad mesh partitions of the surface where the 3D point cloud dataset lies.

Theorems & Definitions (1)

• Proposition 1