Blockwise Flow Matching: Improving Flow Matching Models For Efficient High-Quality Generation

TL;DR

Blockwise Flow Matching (BFM) introduces a temporal segmentation of the generative trajectory, with per-segment velocity blocks that better fit the distinct signal characteristics across timesteps. SemFeat conditions velocity blocks on semantic features aligned with pretrained encoders, while Feature Residual Approximation reduces inference cost by approximating those features on the fly. On ImageNet-256, BFM achieves a substantially improved Pareto frontier, delivering 2.1x to 4.9x faster inference with comparable or superior sample quality, and showing compatibility with mean-flow-based few-step methods and scalability to higher resolutions. Together, these components enable efficient, high-fidelity flow-based generation and offer practical benefits for large-scale generative applications.

Abstract

Recently, Flow Matching models have pushed the boundaries of high-fidelity data generation across a wide range of domains. It typically employs a single large network to learn the entire generative trajectory from noise to data. Despite their effectiveness, this design struggles to capture distinct signal characteristics across timesteps simultaneously and incurs substantial inference costs due to the iterative evaluation of the entire model. To address these limitations, we propose Blockwise Flow Matching (BFM), a novel framework that partitions the generative trajectory into multiple temporal segments, each modeled by smaller but specialized velocity blocks. This blockwise design enables each block to specialize effectively in its designated interval, improving inference efficiency and sample quality. To further enhance generation fidelity, we introduce a Semantic Feature Guidance module that explicitly conditions velocity blocks on semantically rich features aligned with pretrained representations. Additionally, we propose a lightweight Feature Residual Approximation strategy that preserves semantic quality while significantly reducing inference cost. Extensive experiments on ImageNet 256x256 demonstrate that BFM establishes a substantially improved Pareto frontier over existing Flow Matching methods, achieving 2.1x to 4.9x accelerations in inference complexity at comparable generation performance. Code is available at https://github.com/mlvlab/BFM.

Figures (30)

• Figure 1: Blockwise Flow Matching (BFM) achieves a more favorable Pareto-frontier between generation performance (FID) and inference complexity (GFLOPs) on ImageNet $256\times 256$. While standard Flow Matching models train a single model, our BFM partitions the flow trajectory into distinct temporal segments, each handled by a smaller but specialized velocity block, improving generation quality and efficiency.
• Figure 2: Overall pipeline of our method. (a) Blockwise Flow Matching partitions the flow trajectory into $M$ segments, each modeled by a specialized velocity block $v_\theta^{(m)}$. Semantic Feature Guidance enhances the velocity block by explicitly conditioning features $f_t$ from the feature alignment network $f_\phi$. (b) After training velocity models and $f_\phi$, we freeze them and train the Feature Residual Network (FRN) to efficiently approximate $f_t$ by a residual connection. (c) During inference, samples can be efficiently generated by evaluating $f_\phi$ once per segment, reducing inference complexity.
• Figure 3: Spectral entropy (SE) and high-frequency ratio across timesteps.
• Figure 4: Samples from BFM-XL$_\text{SF}$ on ImageNet $256^2$ with classifier-free guidance ($w=4.0$).
• Figure 5: Discrepancy between $f_{t_{m-1}}$ and $f_t$ within each block averaged over 50 samples.