Generative modelling with jump-diffusions

TL;DR

This work broadens score-based diffusion modeling by introducing a jump-diffusion forward process that combines Gaussian noise with finite-activity Lévy jumps, enabling non-Gaussian dynamics and improved tail modeling. It derives a generalized score function $\frac{\nabla \mathcal{V}(\mathbf{x},t)}{p(\mathbf{x},t)}$ and provides both probability-flow ODE and SDE formulations, along with a concrete jump-diffusion Laplace (JDL) model using an OU forward process and isotropic Laplace jump amplitudes. In a heavy-tailed data experiment, the JDL variants outperform heavy-tailed baselines such as LIM and DLPM, demonstrating superior tail accuracy without explicit heavy-tailed priors. The framework opens avenues to other jump statistics and suggests practical extensions like sliced score matching to broaden non-Gaussian diffusion modeling for robust tail behavior in real-world data.

Abstract

Score-based diffusion models generate samples from an unknown target distribution using a time-reversed diffusion process. While such models represent state-of-the-art approaches in industrial applications such as artificial image generation, it has recently been noted that their performance can be further improved by considering injection noise with heavy tailed characteristics. Here, I present a generalization of generative diffusion processes to a wide class of non-Gaussian noise processes. I consider forward processes driven by standard Gaussian noise with super-imposed Poisson jumps representing a finite activity Levy process. The generative process is shown to be governed by a generalized score function that depends on the jump amplitude distribution. Both probability flow ODE and SDE formulations are derived using basic technical effort, and are implemented for jump amplitudes drawn from a multivariate Laplace distribution. Remarkably, for the problem of capturing a heavy-tailed target distribution, the jump-diffusion Laplace model outperforms models driven by alpha-stable noise despite not containing any heavy-tailed characteristics. The framework can be readily applied to other jump statistics that could further improve on the performance of standard diffusion models.