Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Malliavin Calculus for Score-based Diffusion Models

Ehsan Mirafzali, Utkarsh Gupta, Patrick Wyrod, Frank Proske, Daniele Venturi, Razvan Marinescu

TL;DR

This work develops a rigorous Malliavin-calculus framework to obtain exact analytical expressions for the score function $\nabla_y \log p_t(y)$ of forward SDEs underlying score-based diffusion models. By leveraging the Malliavin matrix, first and second variation processes, and a Bismut-type formula, it yields closed-form score representations for linear SDEs that align with Fokker–Planck solutions, and extends to nonlinear SDEs with state-independent diffusion through a Skorokhod integral formulation. The authors provide algorithmic pipelines for both linear and nonlinear cases, including forward-variations simulation, neural estimation of conditional expectations, and reverse-time sampling guided by the Malliavin-derived score. Numerical results on synthetic datasets demonstrate competitive performance with state-of-the-art methods and offer insights into stability and discretisation in nonlinear settings, highlighting the framework's potential to generalise score-based diffusion models beyond Gaussian forward processes. This work grounds score-based modeling in a solid stochastic-analytic foundation, opening pathways to more expressive diffusion models and new estimation strategies for Malliavin derivatives in high dimensions.

Abstract

We introduce a new framework based on Malliavin calculus to derive exact analytical expressions for the score function $\nabla \log p_t(x)$, i.e., the gradient of the log-density associated with the solution to stochastic differential equations (SDEs). Our approach combines classical integration-by-parts techniques with modern stochastic analysis tools, such as Bismut's formula and Malliavin calculus, and it works for both linear and nonlinear SDEs. In doing so, we establish a rigorous connection between the Malliavin derivative, its adjoint, the Malliavin divergence (Skorokhod integral), and diffusion generative models, thereby providing a systematic method for computing $\nabla \log p_t(x)$. In the linear case, we present a detailed analysis showing that our formula coincides with the analytical score function derived from the solution of the Fokker--Planck equation. For nonlinear SDEs with state-independent diffusion coefficients, we derive a closed-form expression for $\nabla \log p_t(x)$. We evaluate the proposed framework across multiple generative tasks and find that its performance is comparable to state-of-the-art methods. These results can be generalised to broader classes of SDEs, paving the way for new score-based diffusion generative models.

Malliavin Calculus for Score-based Diffusion Models

TL;DR

This work develops a rigorous Malliavin-calculus framework to obtain exact analytical expressions for the score function of forward SDEs underlying score-based diffusion models. By leveraging the Malliavin matrix, first and second variation processes, and a Bismut-type formula, it yields closed-form score representations for linear SDEs that align with Fokker–Planck solutions, and extends to nonlinear SDEs with state-independent diffusion through a Skorokhod integral formulation. The authors provide algorithmic pipelines for both linear and nonlinear cases, including forward-variations simulation, neural estimation of conditional expectations, and reverse-time sampling guided by the Malliavin-derived score. Numerical results on synthetic datasets demonstrate competitive performance with state-of-the-art methods and offer insights into stability and discretisation in nonlinear settings, highlighting the framework's potential to generalise score-based diffusion models beyond Gaussian forward processes. This work grounds score-based modeling in a solid stochastic-analytic foundation, opening pathways to more expressive diffusion models and new estimation strategies for Malliavin derivatives in high dimensions.

Abstract

We introduce a new framework based on Malliavin calculus to derive exact analytical expressions for the score function , i.e., the gradient of the log-density associated with the solution to stochastic differential equations (SDEs). Our approach combines classical integration-by-parts techniques with modern stochastic analysis tools, such as Bismut's formula and Malliavin calculus, and it works for both linear and nonlinear SDEs. In doing so, we establish a rigorous connection between the Malliavin derivative, its adjoint, the Malliavin divergence (Skorokhod integral), and diffusion generative models, thereby providing a systematic method for computing . In the linear case, we present a detailed analysis showing that our formula coincides with the analytical score function derived from the solution of the Fokker--Planck equation. For nonlinear SDEs with state-independent diffusion coefficients, we derive a closed-form expression for . We evaluate the proposed framework across multiple generative tasks and find that its performance is comparable to state-of-the-art methods. These results can be generalised to broader classes of SDEs, paving the way for new score-based diffusion generative models.

Paper Structure

This paper contains 34 sections, 19 theorems, 194 equations, 4 figures, 2 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Methodology
  3. Linear SDEs
  4. Nonlinear SDEs with state-independent diffusion
  5. Algorithms
  6. Linear SDEs
  7. Nonlinear SDEs with state-independent diffusion coefficients
  8. Numerical results
  9. Summary
  10. Malliavin calculus
  11. Regularity of densities
  12. Covering vector fields and Bismut-type formula
  13. The first variation process
  14. Malliavin Analysis for Diffusion Models
  15. Malliavin matrix theorem
  16. ...and 19 more sections

Key Result

Theorem 1

The Malliavin matrix $\gamma_{X_T}$ associated with the solution of the linear SDE linSDE at time $T > 0$ is given by

Figures (4)

  • Figure 1: The rows from the top in the diagram show the Checkerboard, 8 Gaussian Mixtures, and Swiss Rolls datasets, and the columns show the ground truth and the results obtained using our framework for the Variance Exploding (VE), Variance Preserving (VP), sub-Variance Preserving (sub-VP) linear SDEs. The last two columns show DDPM and EDM results.
  • Figure 2: Rows (top to bottom) show the Checkerboard, 8 Gaussian Mixtures, and Swiss Rolls datasets, and the columns show the ground truth and the results obtained using our framework for the nonlinear SDEs with Euler (NL-Euler), stochastic Runge–Kutta (NL-SRK), and Predictor–Corrector (NL-PC) integrators.
  • Figure 3: Top row: the SDE’s terminal state with $\beta_{\max}=5$ (left) versus the stationary Cauchy distribution (right), indicating no convergence. Bottom row: the SDE’s terminal state with $\beta_{\max}=25$ (left) versus the stationary Cauchy distribution (right), showing convergence.
  • Figure 4: MNIST results obtained using DDPM (benchmark) and the proposed linear diffusion models based on Malliavin calculus for VE, VP, and sub-VP SDEs.

Theorems & Definitions (39)

  • Definition 1: Malliavin matrix and covering vector field
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3: Score function for linear SDEs
  • proof
  • Theorem 4: Score function for nonlinear SDEs with state-independent diffusion coefficients
  • proof
  • Definition 2
  • ...and 29 more