Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Demystifying Transition Matching: When and Why It Can Beat Flow Matching

Jaihoon Kim, Rajarshi Saha, Minhyuk Sung, Youngsuk Park

TL;DR

The paper provides a principled analysis of Transition Matching (TM) versus Flow Matching (FM) for generative modeling. It proves that in unimodal Gaussian targets, TM yields strictly lower KL divergence for finite steps and can converge faster under a fixed compute budget, due to stochastic difference latent updates that preserve target covariance. Extending to Gaussian mixtures, the authors identify local unimodality regimes where well-separated modes with non-negligible variance allow TM to outperform FM, with approximation errors decaying exponentially with mode separation $D_{\min}$. Empirically, TM shows superior quality–compute trade-offs on Gaussian benchmarks and real-world tasks, including class-conditioned image generation and frame-conditioned video generation, validating its practicality under compute constraints.

Abstract

Flow Matching (FM) underpins many state-of-the-art generative models, yet recent results indicate that Transition Matching (TM) can achieve higher quality with fewer sampling steps. This work answers the question of when and why TM outperforms FM. First, when the target is a unimodal Gaussian distribution, we prove that TM attains strictly lower KL divergence than FM for finite number of steps. The improvement arises from stochastic difference latent updates in TM, which preserve target covariance that deterministic FM underestimates. We then characterize convergence rates, showing that TM achieves faster convergence than FM under a fixed compute budget, establishing its advantage in the unimodal Gaussian setting. Second, we extend the analysis to Gaussian mixtures and identify local-unimodality regimes in which the sampling dynamics approximate the unimodal case, where TM can outperform FM. The approximation error decreases as the minimal distance between component means increases, highlighting that TM is favored when the modes are well separated. However, when the target variance approaches zero, each TM update converges to the FM update, and the performance advantage of TM diminishes. In summary, we show that TM outperforms FM when the target distribution has well-separated modes and non-negligible variances. We validate our theoretical results with controlled experiments on Gaussian distributions, and extend the comparison to real-world applications in image and video generation.

Demystifying Transition Matching: When and Why It Can Beat Flow Matching

TL;DR

The paper provides a principled analysis of Transition Matching (TM) versus Flow Matching (FM) for generative modeling. It proves that in unimodal Gaussian targets, TM yields strictly lower KL divergence for finite steps and can converge faster under a fixed compute budget, due to stochastic difference latent updates that preserve target covariance. Extending to Gaussian mixtures, the authors identify local unimodality regimes where well-separated modes with non-negligible variance allow TM to outperform FM, with approximation errors decaying exponentially with mode separation . Empirically, TM shows superior quality–compute trade-offs on Gaussian benchmarks and real-world tasks, including class-conditioned image generation and frame-conditioned video generation, validating its practicality under compute constraints.

Abstract

Flow Matching (FM) underpins many state-of-the-art generative models, yet recent results indicate that Transition Matching (TM) can achieve higher quality with fewer sampling steps. This work answers the question of when and why TM outperforms FM. First, when the target is a unimodal Gaussian distribution, we prove that TM attains strictly lower KL divergence than FM for finite number of steps. The improvement arises from stochastic difference latent updates in TM, which preserve target covariance that deterministic FM underestimates. We then characterize convergence rates, showing that TM achieves faster convergence than FM under a fixed compute budget, establishing its advantage in the unimodal Gaussian setting. Second, we extend the analysis to Gaussian mixtures and identify local-unimodality regimes in which the sampling dynamics approximate the unimodal case, where TM can outperform FM. The approximation error decreases as the minimal distance between component means increases, highlighting that TM is favored when the modes are well separated. However, when the target variance approaches zero, each TM update converges to the FM update, and the performance advantage of TM diminishes. In summary, we show that TM outperforms FM when the target distribution has well-separated modes and non-negligible variances. We validate our theoretical results with controlled experiments on Gaussian distributions, and extend the comparison to real-world applications in image and video generation.
Paper Structure (43 sections, 16 theorems, 126 equations, 9 figures, 2 tables)

This paper contains 43 sections, 16 theorems, 126 equations, 9 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Overview of Modeling Frameworks
  3. Flow Matching (FM)
  4. Transition Matching (TM)
  5. Scaling Behavior Comparison.
  6. Unimodal Gaussian Target
  7. Convergence Rate of KL Divergence.
  8. KL Divergence Across Discretization Steps.
  9. Analysis of Variance.
  10. Mixture of Gaussians Target
  11. Effect of Mean Separation on Sampling.
  12. KL Divergence Comparison.
  13. Empirical Results
  14. Experiment Setup
  15. Gaussian Synthetic Dataset.
  16. ...and 28 more sections

Key Result

Theorem 1

Let $X_0\sim\mathcal{N}(0,I_d)$ and $X_1\sim\mathcal{N}(\mu,\sigma^2 I_d)$ with $\sigma>0$ be independent Gaussian vectors in $\mathbb{R}^d$. If FM and TM iterates follow $N$ Euler steps with $S>1$ as defined in (eq:fm-euler) and (eq:tm-euler), respectively, then we have where $p_1^{\rm TM}$ and $p_1^{\rm FM}$ are the marginal distributions of $\widetilde{X}_{{t_N}=1}^{\rm TM}$ and $\widetilde{X}

Figures (9)

  • Figure 1: KL Divergence for Unimodal Gaussian Target. Comparison of scaling $S$ at $N=1$ for Transition Matching versus scaling $N$ for Flow Matching. TM achieves lower KL than FM under a fixed compute budget, highlighting the efficacy of increasing $S$.
  • Figure 2: Qualitative Visualization of Unimodal Gaussian Target. Each panel shows the source $\mathcal{N}(0, I_d)$ and target $\mathcal{N}(\mu,\sigma^2 I_d)$ distributions with the generated samples of FM (left) and TM (right). With a small number of steps $(N=2)$, FM produces samples with reduced variance, whereas TM $(N=1, S=2)$ preserves the target variance.
  • Figure 3: Effect of $D_\text{min}$ on $p(V|X)$. Visualization of $p(V|X)$ using cosine-similarity histograms between difference latent samples $\widetilde{V}^{(m)}_{t_n} \sim p(V\mid X_{t_n})$ and $\mathbb{E}[V\mid X_{t_n}]$ for $D_{\min} \in \{8,45\}$. For $D_{\min}=8$ (top) the distribution remains multimodal, whereas for $D_{\min}=45$ (bottom) it concentrates near $1$ at earlier $t_n$, indicating unimodality. A larger $D_{\min}$ tightens Cor. \ref{['cor:localized-bounds']}, so at a fixed $t_n$ the mixture is closer to $p(V|X, Z=k)$.
  • Figure 4: KL Divergence for Mixture of Gaussians Target. Transition Matching (TM) shows lower KL divergence than Flow Matching (FM) as the modes are more separated (red curve, larger $D_\text{min}$). The inset highlights the region near $N = 8$.
  • Figure 5: Quantitative Evaluation v.s. Wall Clock Time. Quantitative comparison of FM (orange) and TM (green) in class-conditioned image generation (left, middle) and frame-conditioned video generation (right), plotted against wall clock time measured in seconds.
  • ...and 4 more figures

Theorems & Definitions (26)

  • Theorem 1
  • Corollary 1
  • Proposition 1
  • Corollary 2
  • Theorem 2
  • Theorem 3
  • proof
  • Corollary
  • proof
  • Proposition
  • ...and 16 more