Diffusion Map Autoencoder

TL;DR

This work presents a Diffusion-Map Autoencoder (DMAE) that fuses a diffusion-map encoder with Nyström out-of-sample extension and two lightweight decoders: a linear ridge map in diffusion coordinates and a Gaussian-Process mean (RBF) latent decoder. The approach yields an inductive, closed-form decoding mechanism suitable for reconstruction and generation without learned convolutions, and it provides a natural uncertainty proxy via the GP decoder. Empirical results on synthetic Swiss-roll data and MNIST-2 show that the RBF-GP decoder offers superior inductive generalization at modest latent dimensions and under noise, while the linear decoder becomes competitive at higher $d$. DMAE delivers a geometrically grounded, sample-efficient alternative to standard autoencoders/VAEs, with clear hyperparameters on the encoder side (bandwidth $\\varepsilon$, normalization $\\alpha$, diffusion time $t$, latent dimension $d$) and a principled latent-space decoding framework; potential extensions include deep-kernel learning and probabilistic variants in the VAE family.

Abstract

Diffusion-Map-AutoEncoder (DMAE) pairs a diffusion-map encoder (using the Nyström method) with linear or RBF Gaussian-Process latent mean decoders, yielding closed-form inductive mappings and strong reconstructions.

Figures (9)

• Figure 1: Linear "PCA" autoencoder.
• Figure 2: Diffusion map encoder architecture.
• Figure 3: DMAE Architectures: Linear Decoder (left) and DMAP ("RBF")-layer decoder (right) .
• Figure 4: Above is an example of the Swiss-roll data set in the ambient (left, with both novel original and reconstructions) and latent coordinates (right).
• Figure 5: We have two series of 3D plots showing the reconstruction of the Swiss-roll. (top) is done with a linear-GP decoder for a series of 6 latent dimensions. (bottom) is done with an RBF-layer.