Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Operator splitting based diffusion samplers and improved convergence analysis

Peiyi Liu, Zhaoqiang Liu, Yiqi Gu

TL;DR

This work introduces operator-splitting samplers for probability flow ODEs in diffusion models by decoupling the linear drift from the nonlinear score-driven drift and composing their flows with Strang splitting. It proves a non-asymptotic total-variation bound that separates discretization, score-mismatch, and Jacobian-mismatch contributions, achieving an overall rate of $O(d/T^{2} + \sqrt{d}\,\varepsilon_{\mathrm{score}} + d\,\varepsilon_{\mathrm{Jac}})$. Under mild regularity, the second-order Strang-splitting sampler attains global trajectory error $O(h^{2})$ and a genuinely second-order TV convergence, with a dimension-linear dependence rather than higher-order scaling seen in some prior works. Experiments on a 2D synthetic dataset corroborate the $O(h^{2})$ scaling and illustrate how score-learning error shifts the TV curve, while preserving the quadratic dependence on the step size. Overall, the paper provides a theoretically principled, training-free yet high-accuracy diffusion sampler framework that blends classical operator-splitting with modern PF-ODE analysis and offers practical guidance for balancing score learning and numerical design.

Abstract

In this paper, we develop a class of samplers for the diffusion model using the operator-splitting technique. The linear drift term and the nonlinear score-driven drift of the probability flow ordinary differential equation are split and applied by flow maps alternatively. Moreover, we conduct detailed analyses for the second-order sampler, establishing a non-asymptotic total variation distance error bound of order $O(d/T^2+\sqrt{d}\varepsilon_{\mathrm{score}}+d\varepsilon_{\mathrm{Jac}})$, where $d$ is the data dimension; $T$ is the number of sampling steps; $\varepsilon_{\mathrm{score}}$ and $\varepsilon_{\mathrm{Jac}}$ measure the discrepancy between the actual score function and learned score function. Our bound is sharper than existing works, yielding bounds of $O(d^p/T^2)$ with some $p>1$ for specific second-order samplers. Numerical experiments on a two-dimensional synthetic dataset corroborate the established quadratic dependence on the step size $1/T$ in the error bound.

Operator splitting based diffusion samplers and improved convergence analysis

TL;DR

This work introduces operator-splitting samplers for probability flow ODEs in diffusion models by decoupling the linear drift from the nonlinear score-driven drift and composing their flows with Strang splitting. It proves a non-asymptotic total-variation bound that separates discretization, score-mismatch, and Jacobian-mismatch contributions, achieving an overall rate of . Under mild regularity, the second-order Strang-splitting sampler attains global trajectory error and a genuinely second-order TV convergence, with a dimension-linear dependence rather than higher-order scaling seen in some prior works. Experiments on a 2D synthetic dataset corroborate the scaling and illustrate how score-learning error shifts the TV curve, while preserving the quadratic dependence on the step size. Overall, the paper provides a theoretically principled, training-free yet high-accuracy diffusion sampler framework that blends classical operator-splitting with modern PF-ODE analysis and offers practical guidance for balancing score learning and numerical design.

Abstract

In this paper, we develop a class of samplers for the diffusion model using the operator-splitting technique. The linear drift term and the nonlinear score-driven drift of the probability flow ordinary differential equation are split and applied by flow maps alternatively. Moreover, we conduct detailed analyses for the second-order sampler, establishing a non-asymptotic total variation distance error bound of order , where is the data dimension; is the number of sampling steps; and measure the discrepancy between the actual score function and learned score function. Our bound is sharper than existing works, yielding bounds of with some for specific second-order samplers. Numerical experiments on a two-dimensional synthetic dataset corroborate the established quadratic dependence on the step size in the error bound.
Paper Structure (22 sections, 13 theorems, 110 equations, 1 figure, 1 table)

This paper contains 22 sections, 13 theorems, 110 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Diffusion Probabilistic Models
  3. Formulation of Samplers
  4. Principle of Operator Splitting
  5. Samplers based on operator splitting
  6. A second-order scheme
  7. Higher-order schemes
  8. Error Analysis
  9. Setting and assumptions
  10. Numerical scheme
  11. Local truncation error
  12. Global error
  13. Total variation error of the sampler
  14. Assumptions
  15. Continuity equations and renormalization
  16. ...and 7 more sections

Key Result

Lemma 1

Let $x(\cdot)$ solve 08. For any $0\le t_{n-1}<t_n\le1$,

Figures (1)

  • Figure 1: Log-log plots of the total variation distance versus the step size $h$ for the Strang--Midpoint sampler: (a) Results using exact score; (b) Results using learned score (a neural network with 2 hidden layers and width 200).

Theorems & Definitions (26)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Lemma 4
  • proof
  • Lemma 5
  • proof
  • ...and 16 more