PCA of probability measures: Sparse and Dense sampling regimes
Gachon Erell, Jérémie Bigot, Elsa Cazelles
TL;DR
The paper tackles PCA on probability measures by embedding measures into Hilbert spaces via KME, LOT, and SW, and analyzes a double asymptotic regime with $n$ measures each observed through $m$ samples. It derives convergence rates for the empirical covariance operator and PCA excess risk of the form $\mathbb{E}\|\hat{\Sigma}-\Sigma\|_{\mathrm{HS}} \lesssim n^{-1/2} + m^{-\alpha}$, where $\alpha>0$ depends on the embedding, revealing a sparse-to-dense transition as $m$ grows relative to $n$ and showing minimax optimality in the dense regime. The PCA excess risk obeys a parallel bound with an embedding-dependent $r_m(\Phi)$, and the authors provide explicit guidance on choosing $m$ given $n$ under different eigenvalue-decay scenarios, including polynomial and exponential decay. Numerical experiments on simulated Gaussian measures and real data (flow cytometry and 3D shapes) validate the theory and demonstrate substantial computational savings from subsampling while preserving PCA accuracy. Overall, the work clarifies how sampling within each measure interacts with the number of measures to govern statistical accuracy in Hilbert-space PCA of distributions, offering practical subsampling strategies for large-scale applications.
Abstract
A common approach to perform PCA on probability measures is to embed them into a Hilbert space where standard functional PCA techniques apply. While convergence rates for estimating the embedding of a single measure from $m$ samples are well understood, the literature has not addressed the setting involving multiple measures. In this paper, we study PCA in a double asymptotic regime where $n$ probability measures are observed, each through $m$ samples. We derive convergence rates of the form $n^{-1/2} + m^{-α}$ for the empirical covariance operator and the PCA excess risk, where $α>0$ depends on the chosen embedding. This characterizes the relationship between the number $n$ of measures and the number $m$ of samples per measure, revealing a sparse (small $m$) to dense (large $m$) transition in the convergence behavior. Moreover, we prove that the dense-regime rate is minimax optimal for the empirical covariance error. Our numerical experiments validate these theoretical rates and demonstrate that appropriate subsampling preserves PCA accuracy while reducing computational cost.