Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Infinite-dimensional generative diffusions via Doob's h-transform

Thorben Pieper-Sethmacher, Daniel Paulin

TL;DR

The paper tackles the challenge of defining and training generative diffusion models directly in infinite dimensions by steering a reference diffusion toward a given target distribution using Doob's $h$-transform. It establishes existence, provides a principled score-matching-based approximation for the steering function, and derives a Wasserstein-2 error bound between the generated and target measures. Specializing to a VP-SPDE, the authors demonstrate robustness to discretisation, enabling sampling in function spaces and in challenging Bayesian inverse problems. Empirical results on Gaussian mixtures, MNIST-SDF, and seismic imaging illustrate competitive fidelity and practical viability, with advantages over time-reversal-based diffusion approaches at finite horizons.

Abstract

This paper introduces a rigorous framework for defining generative diffusion models in infinite dimensions via Doob's h-transform. Rather than relying on time reversal of a noising process, a reference diffusion is forced towards the target distribution by an exponential change of measure. Compared to existing methodology, this approach readily generalises to the infinite-dimensional setting, hence offering greater flexibility in the diffusion model. The construction is derived rigorously under verifiable conditions, and bounds with respect to the target measure are established. We show that the forced process under the changed measure can be approximated by minimising a score-matching objective and validate our method on both synthetic and real data.

Infinite-dimensional generative diffusions via Doob's h-transform

TL;DR

The paper tackles the challenge of defining and training generative diffusion models directly in infinite dimensions by steering a reference diffusion toward a given target distribution using Doob's -transform. It establishes existence, provides a principled score-matching-based approximation for the steering function, and derives a Wasserstein-2 error bound between the generated and target measures. Specializing to a VP-SPDE, the authors demonstrate robustness to discretisation, enabling sampling in function spaces and in challenging Bayesian inverse problems. Empirical results on Gaussian mixtures, MNIST-SDF, and seismic imaging illustrate competitive fidelity and practical viability, with advantages over time-reversal-based diffusion approaches at finite horizons.

Abstract

This paper introduces a rigorous framework for defining generative diffusion models in infinite dimensions via Doob's h-transform. Rather than relying on time reversal of a noising process, a reference diffusion is forced towards the target distribution by an exponential change of measure. Compared to existing methodology, this approach readily generalises to the infinite-dimensional setting, hence offering greater flexibility in the diffusion model. The construction is derived rigorously under verifiable conditions, and bounds with respect to the target measure are established. We show that the forced process under the changed measure can be approximated by minimising a score-matching objective and validate our method on both synthetic and real data.
Paper Structure (27 sections, 11 theorems, 56 equations, 9 figures, 3 tables)

This paper contains 27 sections, 11 theorems, 56 equations, 9 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Challenges and Approach
  3. Contribution
  4. Setup
  5. Forcing via Doob's $h$-transform
  6. Approximation of Doob's h-transform
  7. VP-SPDE
  8. Bounds to the target measure
  9. The case that $\mathop{\mathrm{supp}}\limits(\mu) \subset H_{\nu}$
  10. Applications
  11. Gaussian mixture
  12. MNIST-SDF
  13. Bayesian inverse problem: seismic imaging
  14. Acknowledgements
  15. Proofs for Section \ref{['sec:forcing']}
  16. ...and 12 more sections

Key Result

Theorem 3.3

There exists a unique measure $\mathbb{P}^h$ on $\mathcal{F}_T$, defined by such that $\mathbb{P}^h(X_T \in A) = \mu(A)$ for all $A \in \mathcal{B}(H).$ Moreover, $X$ under $\mathbb{P}^h$ is a mild solution to where $s(t,x) = \mathop{}\!\mathrm{D}_x \log h(t, x)$, $\mathop{}\!\mathrm{d} \mu_0^h(x) = h(0,x) \mathop{}\!\mathrm{d} \mu_0(x)$ and $W^h$ is a $\mathbb{P}^h$-cylindrical Wiener process.

Figures (9)

  • Figure 1: True and generated MNIST-SDF samples (masked) at $64 \times 64$ resolution. (a): True samples, upsampled to $64 \times 64$. (b): Generated samples returned by the VP-SPDE model.
  • Figure 2: Seismic imaging estimate generated by the forced VP-SPDE. (a) Ground truth seismic image $x$. (b) Estimated posterior mean $\hat{x}$. (c) Absolute error between ground truth and posterior mean. (d) Estimated marginal posterior standard deviation.
  • Figure 3: Gaussian mixture samples from the true target distribution defined in \ref{['eq:mixture_gaussian']}.
  • Figure 4: Distances $\mathbb{E}[\|s_{\theta}(t,X_t) - s(t,X_t)\|^2]$ between the true score/steering functions $s(t,x)$ and their approximations $s_{\theta}(t,x)$ for the various generative diffusion models and various state dimensions $D$. (a): the finite-dimensional denoising SDE. (b): the infinite-dimensional denoising SDE. (c): our forced VP-SPDE with constant noise. (c): our forced VP-SPDE with noise scheduling.
  • Figure 5: Gaussian mixture target as defined in \ref{['eq:mixture_gaussian']} sampled through various generative diffusion models. First row: the finite-dimensional denoising SDE. Second row: the infinite-dimensional denoising SDE with constant noise. Third row: our forced VP-SPDE with constant noise. Fourth row: our forced VP-SPDE with noise scheduling. Column one: noising horizon $T = 1$. Column two: noising horizon $T = 0.2$.
  • ...and 4 more figures

Theorems & Definitions (27)

  • Remark 3.2
  • Theorem 3.3
  • Remark 3.4
  • Proposition 4.1
  • Remark 4.2
  • Corollary 4.4
  • Proposition 5.1
  • Corollary 5.2
  • Remark 5.3
  • Remark 5.4
  • ...and 17 more