Transition Matching Distillation for Fast Video Generation

TL;DR

Transition Matching Distillation (TMD) presents a decoupled, two-stage framework to convert large, multi-step video diffusion backbones into efficient few-step generators. By separating a main semantic backbone from a recurrent flow head, and combining Transition Matching with MeanFlow pretraining and distribution-based distillation (DMD2-v) with flow-head rollout, TMD achieves strong speed-quality tradeoffs on Wan2.1 1.3B and 14B video-to-text models. Empirical results show superior visual fidelity and prompt adherence at comparable or reduced compute, including near-one-step generation with high VBench scores. This approach enables practical, real-time video generation while preserving the semantic coherence and detail of large-teacher models.

Abstract

Large video diffusion and flow models have achieved remarkable success in high-quality video generation, but their use in real-time interactive applications remains limited due to their inefficient multi-step sampling process. In this work, we present Transition Matching Distillation (TMD), a novel framework for distilling video diffusion models into efficient few-step generators. The central idea of TMD is to match the multi-step denoising trajectory of a diffusion model with a few-step probability transition process, where each transition is modeled as a lightweight conditional flow. To enable efficient distillation, we decompose the original diffusion backbone into two components: (1) a main backbone, comprising the majority of early layers, that extracts semantic representations at each outer transition step; and (2) a flow head, consisting of the last few layers, that leverages these representations to perform multiple inner flow updates. Given a pretrained video diffusion model, we first introduce a flow head to the model, and adapt it into a conditional flow map. We then apply distribution matching distillation to the student model with flow head rollout in each transition step. Extensive experiments on distilling Wan2.1 1.3B and 14B text-to-video models demonstrate that TMD provides a flexible and strong trade-off between generation speed and visual quality. In particular, TMD outperforms existing distilled models under comparable inference costs in terms of visual fidelity and prompt adherence. Project page: https://research.nvidia.com/labs/genair/tmd

Figures (21)

• Figure 1: Generated examples from TMD. Four frames of 5s 480p videos generated from two text prompts using our TMD method (distilled from Wan2.1 14B T2V) with two different (effective) number of function evaluations (NFE) (see the definition of effective NFE in Eq. \ref{['eq:nfe']}).
• Figure 2: Overview of our TMD method. (a) Decoupled architecture for TMD student, where the main backbone takes the noisy sample ${\bm{x}}_t$, timestep $t$ and text conditioning $c$ as inputs and outputs the main feature ${\bm{m}}_t$, and with a simple fusion layer, flow head conditions on ${\bm{m}}_t, c$ and predicts the less noisy target ${\bm{y}}_r$ from the more noisy ${\bm{y}}_s$ ($s \geq r$). (b) Top: Transition process maps noise to data with a few transition steps. Bottom: In each step, flow head rollout is performed during both distillation and sampling. We omit the timestep inputs $s$ and $r$ to the flow head for simplicity.
• Figure 3: Visual comparison. We compare three frames (and zoomed-in regions of interest) of the outputs of TMD and DMD2-v on exemplary prompts for Wan2.1 1.3B (left) and Wan2.1 14B (right). TMD can improve visual quality at comparable cost to our DMD2-v baseline. Extended prompts can be found in \ref{['app:implementation_details']}.
• Figure 4: Visual comparison. We compare three frames of TMD and DMD2-v on exemplary prompts for Wan2.1 14B. TMD can improve prompt adherence at comparable cost to our DMD2-v baseline. Extended prompts can be found in \ref{['app:implementation_details']}.
• Figure 5: User preference study results. Comparison of TMD-N4H5 (ours) against DMD2-v under one-step ($M=1$) and two-step ($M=2$) distillation regimes. Values indicate the percentage of times users preferred our method over the baseline DMD2-v and the dashed line at 50% represents parity.