Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch

TL;DR

The paper tackles the challenge of slow sampling in large pre-trained flow matching diffusion models by introducing ShortCutting Flow Matching (SCFM), a velocity-space distillation framework that eliminates explicit step-size conditioning. Through a dual-target distillation objective and online self-distillation, SCFM aligns velocity fields across timesteps, enabling ultra-fast few-step samplers without sacrificing quality. Empirically, SCFM distills a 12B Flux teacher to a 3-step sampler in under 24 A100 GPU hours and demonstrates strong performance in few-shot settings with as few as 10 training examples, achieving state-of-the-art fidelity in DeltaFID, FID, and CLIP metrics. This approach offers a practical, data-efficient path to near-zero-cost acceleration of large-scale diffusion models and can be integrated into pretraining, broadening its impact across modalities and domains.

Abstract

We present an ultra-efficient post-training method for shortcutting large-scale pre-trained flow matching diffusion models into efficient few-step samplers, enabled by novel velocity field self-distillation. While shortcutting in flow matching, originally introduced by shortcut models, offers flexible trajectory-skipping capabilities, it requires a specialized step-size embedding incompatible with existing models unless retraining from scratch$\unicode{x2013}$a process nearly as costly as pretraining itself. Our key contribution is thus imparting a more aggressive shortcut mechanism to standard flow matching models (e.g., Flux), leveraging a unique distillation principle that obviates the need for step-size embedding. Working on the velocity field rather than sample space and learning rapidly from self-guided distillation in an online manner, our approach trains efficiently, e.g., producing a 3-step Flux less than one A100 day. Beyond distillation, our method can be incorporated into the pretraining stage itself, yielding models that inherently learn efficient, few-step flows without compromising quality. This capability also enables, to our knowledge, the first few-shot distillation method (e.g., 10 text-image pairs) for dozen-billion-parameter diffusion models, delivering state-of-the-art performance at almost free cost.

Figures (10)

• Figure 1: Visual comparisons on Flux: samples from the original teacher and the TeaCache-accelerated variant uses 32 sampling steps.
• Figure 2: Visual comparisons on SD3.5-Large: samples from the original teacher uses 32 sampling steps.
• Figure 3: CLIP score progression across training methods, evaluated on 20 random image-text pairs from the COCO-30k validation set. Experiments are conducted on Flux and evaluated on the 8-step student.
• Figure 4: Same setting to Figure \ref{['fig:clip-curves']}, focuses exclusively on the convergence behavior during the later stages of training.
• Figure 5: Side-by-side comparison on few-shot trained v.s. regularly trained SCFM. Teacher samples use 32 steps, whereas others use 8.

Theorems & Definitions (2)

• Remark 1
• Remark 2