Predict to Skip: Linear Multistep Feature Forecasting for Efficient Diffusion Transformers

TL;DR

The paper addresses the high computational cost of diffusion transformer inference by exploiting local smoothness in feature trajectories to forecast future outputs instead of naive caching. It introduces PrediT, a training-free framework that casts feature forecasting as a linear multistep problem using a predictor (Adams-Bashforth) and a corrector (Adams-Moulton) with adaptive horizon controlled by a dynamics metric. Empirical results across text-to-image, text-to-video, and class-to-image tasks show substantial speedups up to around 5.5× with negligible quality loss and modest memory overhead, outperforming caching- and Taylor-based baselines. The work demonstrates that principled higher-order prediction with dynamic control can deliver practical, scalable acceleration for diffusion-based generation, with potential extensions to Mixture-of-Experts DiT architectures.

Abstract

Diffusion Transformers (DiT) have emerged as a widely adopted backbone for high-fidelity image and video generation, yet their iterative denoising process incurs high computational costs. Existing training-free acceleration methods rely on feature caching and reuse under the assumption of temporal stability. However, reusing features for multiple steps may lead to latent drift and visual degradation. We observe that model outputs evolve smoothly along much of the diffusion trajectory, enabling principled predictions rather than naive reuse. Based on this insight, we propose \textbf{PrediT}, a training-free acceleration framework that formulates feature prediction as a linear multistep problem. We employ classical linear multistep methods to forecast future model outputs from historical information, combined with a corrector that activates in high-dynamics regions to prevent error accumulation. A dynamic step modulation mechanism adaptively adjusts the prediction horizon by monitoring the feature change rate. Together, these components enable substantial acceleration while preserving generation fidelity. Extensive experiments validate that our method achieves up to $5.54\times$ latency reduction across various DiT-based image and video generation models, while incurring negligible quality degradation.

Figures (10)

• Figure 1: Accelerating Diffusion Transformer inference across multiple models. PrediT achieves significant speedup on various DiT-based architectures for both image and video generation while preserving visual quality.
• Figure 2: Analysis of direct feature reuse limitations. (a) Feature trajectory visualization via PCA shows local smoothness along the diffusion process. (b) Comparison of per-step approximation error and accumulated drift: naive reuse incurs significant error accumulation, while higher-order prediction methods reduce drift.
• Figure 3: Overview of the PrediT framework. PrediT formulates feature prediction as a linear multistep problem, using historical model outputs to extrapolate future features and reduce redundant model calls. The threshold $\tau$ and correction ratio $r$ control the selection between ABM for high-dynamics and AB for low-dynamics regions.
• Figure 4: Feature dynamics vary across the diffusion process. (a) Initial and final phases exhibit rapid changes. (b) Approximation error grows with skip interval, especially in high-dynamics regions.
• Figure 5: Visual quality comparison of different acceleration methods on FLUX. PrediT achieves higher acceleration while maintaining comparable visual quality.

Theorems & Definitions (2)

• Proposition 2.1: AB Local Truncation Error
• Proposition 2.2: Error Bound with Dynamic Step Modulation