Model-free filtering in high dimensions via projection and score-based diffusions
Sören Christensen, Jan Kallsen, Claudia Strauch, Lukas Trottner
TL;DR
This work develops a model-free, geometry-driven approach to denoise and project high-dimensional data onto an unknown low-dimensional manifold $\mathscr M$. By embedding an auxiliary diffusion model with isotropic Laplace-like noise into a killed Brownian framework, the authors derive a posterior sampler for $\mathbb{P}^{X|Y}$ whose backward drift $b(y)$ is learned via score matching. In high dimensions, they prove that the posterior concentrates near the metric projection of $Y$ onto $\mathscr M$, effectively realizing a probabilistic nearest-neighbor projection onto the data manifold. The methodology is demonstrated on image-like data, with a MNIST-inspired experiment showing the practicality of projecting noisy observations onto a learned data manifold through a time-reversed diffusion with a random horizon. This provides a robust, data-driven denoising paradigm that leverages manifold structure without requiring explicit noise modeling.
Abstract
We consider the problem of recovering a latent signal $X$ from its noisy observation $Y$. The unknown law $\mathbb{P}^X$ of $X$, and in particular its support $\mathscr{M}$, are accessible only through a large sample of i.i.d.\ observations. We further assume $\mathscr{M}$ to be a low-dimensional submanifold of a high-dimensional Euclidean space $\mathbb{R}^d$. As a filter or denoiser $\widehat X$, we suggest an estimator of the metric projection $π_{\mathscr{M}}(Y)$ of $Y$ onto the manifold $\mathscr{M}$. To compute this estimator, we study an auxiliary semiparametric model in which $Y$ is obtained by adding isotropic Laplace noise to $X$. Using score matching within a corresponding diffusion model, we obtain an estimator of the Bayesian posterior $\mathbb{P}^{X \mid Y}$ in this setup. Our main theoretical results show that, in the limit of high dimension $d$, this posterior $\mathbb{P}^{X\mid Y}$ is concentrated near the desired metric projection $π_{\mathscr{M}}(Y)$.