Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Telegrapher's Generative Model via Kac Flows

Richard Duong, Jannis Chemseddine, Peter K. Friz, Gabriele Steidl

TL;DR

The paper introduces a telegrapher's equation–based generative model (Kac flow) as a bounded-velocity alternative to diffusion for flow-based generation. It proves Lipschitz regularity of the probability flow in Wasserstein space, derives analytic conditional velocity fields for 1D Dirac starts, and shows convergence to diffusion in appropriate limits; it further extends to multi-D via independent coordinate-wise Kac processes with explicit velocity decompositions. A neural network is trained via conditional flow matching to approximate the velocity fields, leveraging the decomposed structure for scalable learning and sampling. Numerical experiments demonstrate robustness to velocity explosions, improved recovery of Dirac-like modes in 2D, and competitive image generation on CIFAR-10, highlighting practical advantages over diffusion-based methods. Overall, the framework provides a principled, physics-inspired, and scalable approach to flow-based generations with bounded velocities and explicit velocity-field structure.

Abstract

We break the mold in flow-based generative modeling by proposing a new model based on the damped wave equation, also known as telegrapher's equation. Similar to the diffusion equation and Brownian motion, there is a Feynman-Kac type relation between the telegrapher's equation and the stochastic Kac process in 1D. The Kac flow evolves stepwise linearly in time, so that the probability flow is Lipschitz continuous in the Wasserstein distance and, in contrast to diffusion flows, the norm of the velocity is globally bounded. Furthermore, the Kac model has the diffusion model as its asymptotic limit. We extend these considerations to a multi-dimensional stochastic process which consists of independent 1D Kac processes in each spatial component. We show that this process gives rise to an absolutely continuous curve in the Wasserstein space and compute the conditional velocity field starting in a Dirac point analytically. Using the framework of flow matching, we train a neural network that approximates the velocity field and use it for sample generation. Our numerical experiments demonstrate the scalability of our approach, and show its advantages over diffusion models.

Telegrapher's Generative Model via Kac Flows

TL;DR

The paper introduces a telegrapher's equation–based generative model (Kac flow) as a bounded-velocity alternative to diffusion for flow-based generation. It proves Lipschitz regularity of the probability flow in Wasserstein space, derives analytic conditional velocity fields for 1D Dirac starts, and shows convergence to diffusion in appropriate limits; it further extends to multi-D via independent coordinate-wise Kac processes with explicit velocity decompositions. A neural network is trained via conditional flow matching to approximate the velocity fields, leveraging the decomposed structure for scalable learning and sampling. Numerical experiments demonstrate robustness to velocity explosions, improved recovery of Dirac-like modes in 2D, and competitive image generation on CIFAR-10, highlighting practical advantages over diffusion-based methods. Overall, the framework provides a principled, physics-inspired, and scalable approach to flow-based generations with bounded velocities and explicit velocity-field structure.

Abstract

We break the mold in flow-based generative modeling by proposing a new model based on the damped wave equation, also known as telegrapher's equation. Similar to the diffusion equation and Brownian motion, there is a Feynman-Kac type relation between the telegrapher's equation and the stochastic Kac process in 1D. The Kac flow evolves stepwise linearly in time, so that the probability flow is Lipschitz continuous in the Wasserstein distance and, in contrast to diffusion flows, the norm of the velocity is globally bounded. Furthermore, the Kac model has the diffusion model as its asymptotic limit. We extend these considerations to a multi-dimensional stochastic process which consists of independent 1D Kac processes in each spatial component. We show that this process gives rise to an absolutely continuous curve in the Wasserstein space and compute the conditional velocity field starting in a Dirac point analytically. Using the framework of flow matching, we train a neural network that approximates the velocity field and use it for sample generation. Our numerical experiments demonstrate the scalability of our approach, and show its advantages over diffusion models.

Paper Structure

This paper contains 29 sections, 11 theorems, 100 equations, 11 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Outline of the paper.
  4. Preliminaries: Curves in Wasserstein Spaces
  5. Motivation
  6. 1. Flows from couplings.
  7. 2. Diffusion flows.
  8. 3. Flows from linear interpolation of Boltzmann energies.
  9. 4. Poisson flows.
  10. Telegrapher's Equation and Kac Model in 1D
  11. Kac Flow in 1D
  12. Relation of Kac' Flow to 1D Telegrapher's Equation
  13. Multi-dimensional Generalizations of the Telegrapher's Model
  14. Kac's Insertion Method
  15. Multi-Dimensional Kac Process
  16. ...and 14 more sections

Key Result

Theorem 1

For any initial function $f_0 \in H^2(\mathbb{R})$, the function solves the 1D telegrapher's equation tele-eq-1D. Further, if $f_0$ is a probability density of a random variable $X_0$ independent of $S(t)$, then tele-sol-1D is the probability density of the Kac process $(X_t)_{t \ge 0}$ starting in $X_0 \sim f_0$ defined by eq:kac_1.

Figures (11)

  • Figure 1: Paths of the componentwise Kac walk in 2D, simulated until time $T=10$ with damping/velocity parameters $(a,c) = (1,1), (2,1 ), (4,2), (25,5)$, and a standard Brownian motion (right).
  • Figure 2: The distribution of the Kac process $\mathbf{X}_t$ starting in a 2D Gaussian mixture, simulated until time $T=10$ with $(a,c) =(1,1)$ (upper row) and $(a,c) = (5,5)$ (lower row).
  • Figure 3: The distribution of $\mathbf{X}_t$ starting in a 2D Gaussian mixture, simulated until time $T=3$ with 'large' parameters $(a,c) =(100,10)$. As described in Theorem \ref{['convergence-multi-d']}, we see a 'diffusion' like process with approximate variance $\sigma^2 t = \frac{c^2}{a}t = t$.
  • Figure 4: Backward evolution of the learned Kac flow for $(a,c) =(25,5)$, see also Figure \ref{['fig:2d_experiment']}.
  • Figure 5: Generated samples (blue) vs. ground truth (red) at the indicated iteration for each model. The Kac model can precisely recover the small modes, while the diffusion model creates "blobs".
  • ...and 6 more figures

Theorems & Definitions (22)

  • Theorem 1
  • proof
  • Lemma 2
  • Remark 3
  • Theorem 4
  • proof
  • Theorem 5
  • Remark 6
  • Lemma 7
  • Theorem 8
  • ...and 12 more