Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Fractional Diffusion Models

Gabriel Nobis, Maximilian Springenberg, Marco Aversa, Michael Detzel, Rembert Daems, Roderick Murray-Smith, Shinichi Nakajima, Sebastian Lapuschkin, Stefano Ermon, Tolga Birdal, Manfred Opper, Christoph Knochenhauer, Luis Oala, Wojciech Samek

TL;DR

GFDM is introduced, the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics and achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID.

Abstract

We introduce the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics. Although diffusion models have excelled at capturing data distributions, they still suffer from various limitations such as slow convergence, mode-collapse on imbalanced data, and lack of diversity. These issues are partially linked to the use of light-tailed Brownian motion (BM) with independent increments. In this paper, we replace BM with an approximation of its non-Markovian counterpart, fractional Brownian motion (fBM), characterized by correlated increments and Hurst index $H \in (0,1)$, where $H=0.5$ recovers the classical BM. To ensure tractable inference and learning, we employ a recently popularized Markov approximation of fBM (MA-fBM) and derive its reverse-time model, resulting in generative fractional diffusion models (GFDM). We characterize the forward dynamics using a continuous reparameterization trick and propose augmented score matching to efficiently learn the score function, which is partly known in closed form, at minimal added cost. The ability to drive our diffusion model via MA-fBM offers flexibility and control. $H \leq 0.5$ enters the regime of rough paths whereas $H>0.5$ regularizes diffusion paths and invokes long-term memory. The Markov approximation allows added control by varying the number of Markov processes linearly combined to approximate fBM. Our evaluations on real image datasets demonstrate that GFDM achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID, offering a promising alternative to traditional diffusion models

Generative Fractional Diffusion Models

TL;DR

GFDM is introduced, the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics and achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID.

Abstract

We introduce the first continuous-time score-based generative model that leverages fractional diffusion processes for its underlying dynamics. Although diffusion models have excelled at capturing data distributions, they still suffer from various limitations such as slow convergence, mode-collapse on imbalanced data, and lack of diversity. These issues are partially linked to the use of light-tailed Brownian motion (BM) with independent increments. In this paper, we replace BM with an approximation of its non-Markovian counterpart, fractional Brownian motion (fBM), characterized by correlated increments and Hurst index , where recovers the classical BM. To ensure tractable inference and learning, we employ a recently popularized Markov approximation of fBM (MA-fBM) and derive its reverse-time model, resulting in generative fractional diffusion models (GFDM). We characterize the forward dynamics using a continuous reparameterization trick and propose augmented score matching to efficiently learn the score function, which is partly known in closed form, at minimal added cost. The ability to drive our diffusion model via MA-fBM offers flexibility and control. enters the regime of rough paths whereas regularizes diffusion paths and invokes long-term memory. The Markov approximation allows added control by varying the number of Markov processes linearly combined to approximate fBM. Our evaluations on real image datasets demonstrate that GFDM achieves greater pixel-wise diversity and enhanced image quality, as indicated by a lower FID, offering a promising alternative to traditional diffusion models
Paper Structure (47 sections, 6 theorems, 98 equations, 12 figures, 8 tables)

This paper contains 47 sections, 6 theorems, 98 equations, 12 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Differentiation from existing work
  3. Background
  4. Simulation error of the reverse-time model
  5. A pathwise perspective on sampling
  6. Fractional driving noise
  7. Generalization challenges
  8. A score-based generative model based on fractional noise
  9. Derivation of marginal statistics
  10. Explicit fractional forward dynamics
  11. The reverse-time model
  12. Augmented score matching
  13. Sampling from reverse-time model
  14. Experiments
  15. Effect of augmentation on MNIST
  16. ...and 32 more sections

Key Result

Proposition 3.3

The optimal approximation coefficients $\bm{\omega}=(\omega_{1},...,\omega_{K})\in\mathbb{R}^{K}$ for a given Hurst index $H\in(0,1)$, a terminal time $T>0$ and a fixed geometrically spaced grid to minimize the $L^{2}(\mathbb{P})$-error are given by the closed-form expression $\bm{A}\bm{\omega} = \bm{b}$ with and where $P(z,x)=\frac{1}{\Gamma(z)}\int^{x}_{0}t^{z-1}e^{-t}\mathrm{d} t$ is the regu

Figures (12)

  • Figure 1: Each data dimension transitions to a known prior distribution through a forward process that approximates a fractional diffusion process. The Hurst index $H$ on the LHS interpolates between the roughness of a Brownian driven SDE and the underlying integration in PF ODEs. The driving noise process is a linear combination of the correlated processes on the RHS, all driven by the same Brownian motion. The score function of these augmenting processes is available in closed form and serves as guidance for the unknown score function.
  • Figure 2: Comparison of the super-diffusive regime and purely Brownian dynamics in terms of average FID over three rounds of sampling plotted across different NFEs.
  • Figure 3: Quantitative performance comparison of SDE and PF ODE sampling.
  • Figure 4: Visual comparison of PF ODE samples. (LHS) Purely Brownian VP dynamics. (RHS) Super-diffusive regime $\text{FVP}(H=0.9, K=2)$.
  • Figure 5: Analytical solution (blue) used by our method for FVE dynamics with $K=5$ and $H=0.5$ compared to the approximated solution (dashed red) resulting from solving ODE (\ref{['eq:ode_cov']}).
  • ...and 7 more figures

Theorems & Definitions (13)

  • Definition 3.1: Type ii Fractional Brownian Motion levy1953random
  • Definition 3.2: Markov approximation of fBM HARMS2019daems2023variational
  • Proposition 3.3: Optimal Approximation Coefficients daems2023variational
  • Definition 4.1: Forward process
  • Proposition 4.2: Continuous Reparameterization Trick
  • Proposition 4.3: Optimal Score Model
  • Remark 4.4
  • Definition A.1: Type i Fractional Brownian Motion mandelbrot1968fractional
  • Theorem A.2: Markovian representation of fBM HARMS2019daems2023variational
  • Proposition A.3: Continuous Reparameterization Trick
  • ...and 3 more