Simple, Fast and Efficient Injective Manifold Density Estimation with Random Projections
Ahmad Ayaz Amin, Baha Uddin Kazi
TL;DR
This work addresses density estimation with injective flows by introducing Random Projection Flows (RPFs), which compress high-dimensional data into a lower-dimensional latent space via Haar-distributed semi-orthogonal projections. A key advantage is the closed-form, constant Riemannian volume correction term $\log \sqrt{\det(J^\top J)} = \frac{d}{2} \log \frac{D}{d}$, with an optional Johnson–Lindenstrauss scaling to improve calibration. RPFs are data-agnostic, composable within broader NF architectures, and empirically competitive with PCA-based injective flows on UCI benchmarks and synthetic manifolds, while highlighting generation challenges on complex datasets like CIFAR-10 when using simple latent models. The results illustrate that preserving intrinsic manifold geometry via random projections can enhance likelihood-based density estimation, offering a low-cost, principled baseline and a bridge between random-matrix theory and normalizing flows, with future work pointing to richer latent densities and deeper projections to scale to harder data.
Abstract
We introduce Random Projection Flows (RPFs), a principled framework for injective normalizing flows that leverages tools from random matrix theory and the geometry of random projections. RPFs employ random semi-orthogonal matrices, drawn from Haar-distributed orthogonal ensembles via QR decomposition of Gaussian matrices, to project data into lower-dimensional latent spaces for the base distribution. Unlike PCA-based flows or learned injective maps, RPFs are plug-and-play, efficient, and yield closed-form expressions for the Riemannian volume correction term. We demonstrate that RPFs are both theoretically grounded and practically effective, providing a strong baseline for generative modeling and a bridge between random projection theory and normalizing flows.