MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference

TL;DR

MeanCache tackles the instability of caching-based acceleration in Flow Matching by shifting from instantaneous velocity caching to an average-velocity framework and coupling it with a JVP-based correction. By defining the interval mean velocity $u(z_t,t,s)$ and leveraging a start-point identity alongside a reference-point JVP estimator, the method reconstructs stable trajectories without retraining. A trajectory-stability scheduling algorithm—built on a graph of timesteps and a peak-suppressed shortest-path objective—optimally places caches under budget constraints. Experiments on FLUX.1, Qwen-Image, and HunyuanVideo demonstrate 3–4x speedups with consistent or improved generation quality, highlighting MeanCache’s practical potential for commercial-scale generative models.

Abstract

We present MeanCache, a training-free caching framework for efficient Flow Matching inference. Existing caching methods reduce redundant computation but typically rely on instantaneous velocity information (e.g., feature caching), which often leads to severe trajectory deviations and error accumulation under high acceleration ratios. MeanCache introduces an average-velocity perspective: by leveraging cached Jacobian--vector products (JVP) to construct interval average velocities from instantaneous velocities, it effectively mitigates local error accumulation. To further improve cache timing and JVP reuse stability, we develop a trajectory-stability scheduling strategy as a practical tool, employing a Peak-Suppressed Shortest Path under budget constraints to determine the schedule. Experiments on FLUX.1, Qwen-Image, and HunyuanVideo demonstrate that MeanCache achieves 4.12X and 4.56X and 3.59X acceleration, respectively, while consistently outperforming state-of-the-art caching baselines in generation quality. We believe this simple yet effective approach provides a new perspective for Flow Matching inference and will inspire further exploration of stability-driven acceleration in commercial-scale generative models.

Figures (14)

• Figure 1: Visualization of images generated by different methods on FLUX.1[dev] under varying acceleration ratios.
• Figure 2: Instantaneous vs. Average Velocity and JVP Caching.(Left) Along the original trajectory, instantaneous velocity shows sharp fluctuations, while average velocity is much smoother. (Middle) At timestep 927, JVP Caching reduces error accumulation, though its effectiveness depends on the cache interval and hyperparameter $K$. (Right) At timestep 551, it achieves stronger error mitigation, showing that effectiveness varies across timesteps. Both middle and right figures are under the single-cache setting on the original trajectory.
• Figure 3: From Instantaneous to Average Velocity. Directly caching the instantaneous velocity $v(z_t,t)$ over $[t,s]$ easily leads to trajectory drift and error accumulation, whereas the average velocity $u(z_t,t,s)$ accurately reaches the target $s$. MeanCache introduces a prior timestep $r$ and reuses $\mathrm{JVP}_{r \to t}$ to estimate the average velocity $\hat{u}(z_t,t,s)$, thereby correcting the trajectory and effectively mitigating error accumulation.
• Figure 3: Impact of peak-suppression parameter $\gamma$ on quality metrics.
• Figure 4: Comparison of different methods at high acceleration ratios on FLUX.1[dev].