FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generation

TL;DR

FlowBlending addresses the expensive cost of diffusion-based video generation by revealing that model capacity is most impactful during the early and late denoising stages, while the intermediate stage is capacity-tolerant. It introduces a simple, training-free stage-aware sampling strategy (LSL) that uses a large model for early structure formation and late refinement and a small model for the middle stage, achieving up to 1.65× inference speed and 57.35% fewer FLOPs while preserving large-model fidelity. The approach is validated on LTX-Video and WAN 2.1, demonstrating strong parity with large-only outputs across FID, FVD, and VBench metrics, and it is shown to be compatible with other acceleration methods like step-reduction and distillation. Velocity-divergence analysis and DINO/CLIP-based boundary detection provide practical, model-agnostic cues for identifying stage boundaries, enabling robust, near-optimal trade-offs in diverse configurations.

Abstract

In this work, we show that the impact of model capacity varies across timesteps: it is crucial for the early and late stages but largely negligible during the intermediate stage. Accordingly, we propose FlowBlending, a stage-aware multi-model sampling strategy that employs a large model and a small model at capacity-sensitive stages and intermediate stages, respectively. We further introduce simple criteria to choose stage boundaries and provide a velocity-divergence analysis as an effective proxy for identifying capacity-sensitive regions. Across LTX-Video (2B/13B) and WAN 2.1 (1.3B/14B), FlowBlending achieves up to 1.65x faster inference with 57.35% fewer FLOPs, while maintaining the visual fidelity, temporal coherence, and semantic alignment of the large models. FlowBlending is also compatible with existing sampling-acceleration techniques, enabling up to 2x additional speedup. Project page is available at: https://jibin86.github.io/flowblending_project_page.

Figures (19)

• Figure 1: Overview of FlowBlending. The videos in each column are generated from the same initial noise and text prompt, but with different model allocation strategies. FlowBlending assigns a large model to process early and late denoising stages---establishing global structure and refining details, respectively--- and assigns a small model to process intermediate denoising stages, where velocity divergence between the two models is minimal. This approach preserves visual fidelity of the large model while reducing computation.
• Figure 2: Effect of model capacity during early denoising stage. Comparison of WAN-2.1 sampling schedules (L = 14B, S = 1.3B). LSS (large only in early steps) closely matches LLL (large-only) in structure and motion, while SSS (small-only) exhibits temporal inconsistency and semantic misalignment. SLL (small only in early steps) likewise produces structure and motion patterns highly similar to SSS. This shows that the early stages are crucial for establishing global semantic and structural attributes.
• Figure 3: Late stage ablation: LLL vs. LSS vs. LSL. LSS preserves global structure similar to LLL but exhibits some artifacts. Reintroducing the large model only during the late stage (LSL) restores detail and reduces flicker, demonstrating that the late denoising stage is capacity-sensitive. Notably, the LSL schedule attains quality nearly indistinguishable from LLL while retaining the efficiency benefits of using the small model for most of the trajectory. Please zoom in to view the figures in detail.
• Figure 4: DINO-based identification of the early stage boundary. We measure the frame-wise DINO similarity between schedules that switch from the large to the small model ($L \rightarrow S$) at different early stage boundaries and the large-only baseline (LLL). A sharp decline in similarity emerges beyond a specific point in the curve. Boundaries chosen just before this drop (typically above $\sim$96% similarity) preserve global structure and motion.
• Figure 5: FID-based identification of the late stage boundary. We fix the early stage boundary identified in Section \ref{['sec:early_stage_quantitative']} and vary only the late stage boundary. The resulting FID curve exhibits a V-shape, where the minimum corresponds to the optimal late stage boundary used in our LSL schedule. This trend consistently appears in both WAN and LTX-Video, demonstrating that a properly chosen late boundary yields the best sweet spot with detail refinement and artifact suppression.