Infinite-Dimensional Diffusion Models

TL;DR

This work advances diffusion modeling by formulating and analyzing diffusion processes directly in infinite-dimensional Hilbert spaces, enabling principled function‑space generative modeling. It resolves key theoretical challenges—defining an infinite‑dimensional score via conditional expectations, ensuring well‑posed forward/reverse SDEs, and obtaining dimension‑free convergence guarantees—while providing practical guidelines for noise covariances and loss norms. The authors introduce two design paradigms (IDDM1 and IDDM2) and show that for image data the canonical White Noise Diffusion Model aligns with the theory, whereas other data distributions benefit from tailored choices. They validate the framework theoretically with existence/uniqueness and Wasserstein bounds and empirically across function‑space examples including manifolds and Bayesian inverse problems. The work thus offers a principled, scalable path for diffusion modeling directly in function spaces, with implications for inverse problems, simulations, and other infinite‑dimensional data domains.

Abstract

Diffusion models have had a profound impact on many application areas, including those where data are intrinsically infinite-dimensional, such as images or time series. The standard approach is first to discretize and then to apply diffusion models to the discretized data. While such approaches are practically appealing, the performance of the resulting algorithms typically deteriorates as discretization parameters are refined. In this paper, we instead directly formulate diffusion-based generative models in infinite dimensions and apply them to the generative modelling of functions. We prove that our formulations are well posed in the infinite-dimensional setting and provide dimension-independent distance bounds from the sample to the target measure. Using our theory, we also develop guidelines for the design of infinite-dimensional diffusion models. For image distributions, these guidelines are in line with current canonical choices. For other distributions, however, we can improve upon these canonical choices. We demonstrate these results both theoretically and empirically, by applying the algorithms to data distributions on manifolds and to distributions arising in Bayesian inverse problems or simulation-based inference.

Figures (8)

• Figure 1: In each panel, we plot samples from $\pi^\alpha$ for different values of $\alpha$, where $\pi^\alpha$ is defined in Section \ref{['sec:family_gaussian_measures']}. We chose $e_k(\cdot) = \sqrt{2}\sin(2 \pi k \, \cdot \, )$ as an orthonormal basis of $L^2$. For $\alpha=0$ we see a sample of space-time white noise, where no function value is correlated to any of its neighboring function values. For $\alpha = 1$ and our specific choice of $e_k$, the sampled measure is the Brownian bridge measure.
• Figure 2: We generated $50~000$ training examples from the distribution described in Section \ref{['sec:numerics_sphere']}. On the left, we show a heatmap of the resulting marginal densities of function values at each point in the domain $[0,1]$. On the right, we plot a few training samples.
• Figure 3: Example of Section \ref{['sec:numerics_sphere']}: samples generated by WNDM (row 1) and IDDM1 (row 2) after increasing numbers of training epochs. Samples from the true measure can be compared in Figure \ref{['fig:sphere_256D_train']}.
• Figure 4: Example of Section \ref{['sec:numerics_sphere']}: each vertical slice shows a heatmap of the marginal density estimated from 2048 samples generated by each of the diffusion models, after 60 epochs of training. For comparison, the heatmap of the 50 000 training examples is plotted in Figure \ref{['fig:sphere_256D_train']}. The one-dimensional marginals are matched well by both algorithms.
• Figure 5: Example of Section \ref{['sec:volatility_estimation']}. As a reference/comparison, we generate $50\,000$ high-quality posterior samples from $d \pi^{\alpha_\text{data}}(a_\tau \vert \tilde{r})$ using the Hilbert space Hamiltonian Monte Carlo algorithm. On the left is a heatmap of posterior marginal densities of $a_\tau$, at each point in the domain $\tau \in [0,1]$. On the right, we plot a few example posterior samples.

Theorems & Definitions (18)

• Lemma 1
• Definition 2
• Remark 3
• Remark 4
• Lemma 5
• Remark 6
• Lemma 7
• Lemma 8
• Theorem 9
• Remark 10