Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Malliavin calculus approach to score functions in diffusion generative models

Ehsan Mirafzali, Frank Proske, Utkarsh Gupta, Daniele Venturi, Razvan Marinescu

TL;DR

The paper addresses the challenge of obtaining a tractable score function for nonlinear diffusion processes used in score-based diffusion models. It develops a closed-form, Malliavin-calculus–based representation of the score $\nabla_y \log p_T(y)$ as $-\mathbb{E}[\delta(u_k)\mid X_T=y]$, where $\delta(u_k)$ is a Skorokhod integral expressed entirely through first and second variation processes, thus eliminating explicit Malliavin derivatives from the final formula. The main contribution is a novel BEL-type formula that converts abstract Malliavin objects into pathwise, computable quantities, plus a state-independent diffusion simplification relevant for common diffusion schedulers. This framework provides a principled foundation for designing new sampling and score-estimation algorithms for nonlinear SDEs, with potential impact on high-fidelity data generation and applications requiring nonlinear diffusion dynamics. The results extend to broader diffusion classes and offer a robust, implementable route to accurate score estimation in nonlinear diffusion-based generative modelling.

Abstract

Score-based diffusion generative models have recently emerged as a powerful tool for modelling complex data distributions. These models aim at learning the score function, which defines a map from a known probability distribution to the target data distribution via deterministic or stochastic differential equations (SDEs). The score function is typically estimated from data using a variety of approximation techniques, such as denoising or sliced score matching, Hyvärien's method, or Schrödinger bridges. In this paper, we derive an exact, closed-form, expression for the score function for a broad class of nonlinear diffusion generative models. Our approach combines modern stochastic analysis tools such as Malliavin derivatives and their adjoint operators (Skorokhod integrals or Malliavin Divergence) with a new Bismut-type formula. The resulting expression for the score function can be written entirely in terms of the first and second variation processes, with all Malliavin derivatives systematically eliminated, thereby enhancing its practical applicability. The theoretical framework presented in this work offers a principled foundation for advancing score estimation methods in generative modelling, enabling the design of new sampling algorithms for complex probability distributions. Our results can be extended to broader classes of stochastic differential equations, opening new directions for the development of score-based diffusion generative models.

A Malliavin calculus approach to score functions in diffusion generative models

TL;DR

The paper addresses the challenge of obtaining a tractable score function for nonlinear diffusion processes used in score-based diffusion models. It develops a closed-form, Malliavin-calculus–based representation of the score as , where is a Skorokhod integral expressed entirely through first and second variation processes, thus eliminating explicit Malliavin derivatives from the final formula. The main contribution is a novel BEL-type formula that converts abstract Malliavin objects into pathwise, computable quantities, plus a state-independent diffusion simplification relevant for common diffusion schedulers. This framework provides a principled foundation for designing new sampling and score-estimation algorithms for nonlinear SDEs, with potential impact on high-fidelity data generation and applications requiring nonlinear diffusion dynamics. The results extend to broader diffusion classes and offer a robust, implementable route to accurate score estimation in nonlinear diffusion-based generative modelling.

Abstract

Score-based diffusion generative models have recently emerged as a powerful tool for modelling complex data distributions. These models aim at learning the score function, which defines a map from a known probability distribution to the target data distribution via deterministic or stochastic differential equations (SDEs). The score function is typically estimated from data using a variety of approximation techniques, such as denoising or sliced score matching, Hyvärien's method, or Schrödinger bridges. In this paper, we derive an exact, closed-form, expression for the score function for a broad class of nonlinear diffusion generative models. Our approach combines modern stochastic analysis tools such as Malliavin derivatives and their adjoint operators (Skorokhod integrals or Malliavin Divergence) with a new Bismut-type formula. The resulting expression for the score function can be written entirely in terms of the first and second variation processes, with all Malliavin derivatives systematically eliminated, thereby enhancing its practical applicability. The theoretical framework presented in this work offers a principled foundation for advancing score estimation methods in generative modelling, enabling the design of new sampling algorithms for complex probability distributions. Our results can be extended to broader classes of stochastic differential equations, opening new directions for the development of score-based diffusion generative models.

Paper Structure

This paper contains 9 sections, 12 theorems, 44 equations.

Table of Contents

  1. Introduction
  2. Problem statement and main result
  3. Malliavin calculus and covering vector fields
  4. Variation processes
  5. Proof of Theorem \ref{['thm:main']}
  6. Useful lemmas
  7. Proof of Theorem \ref{['thm:main']}
  8. State-independent diffusion processes
  9. References

Key Result

Theorem 2.1

Let $X_t$ be the solution to the stochastic differential equation where $b: [0, T] \times \mathbb{R}^m \to \mathbb{R}^m$ and $\sigma: [0, T] \times \mathbb{R}^m \to \mathbb{R}^{m \times d}$ are sufficiently smooth functions, and $B_t$ is a $d$-dimensional Brownian motion. Define the first variation process and the Malliavin covariance matrix of $X_T$ as which we assume to be invertible almost s

Theorems & Definitions (13)

  • Theorem 2.1: Skorokhod integral representation theorem for the score function
  • Theorem 3.1: Existence and uniqueness of the covering vector field
  • Theorem 3.2: Regularity of the covering vector field
  • Theorem 3.3: $L^2$–continuity of the covering vector field
  • Theorem 3.4: Stability under perturbations
  • Theorem 5.1: Theorem 3.2.9, nualart2006malliavin
  • Lemma 5.2: SDE for the inverse first variation process
  • Lemma 5.3: Malliavin derivative of inverse matrices
  • Remark 5.4: The role of integrability and ellipticity conditions
  • Lemma 5.5: Commutativity of Malliavin and partial derivatives
  • ...and 3 more