DiCache: Let Diffusion Model Determine Its Own Cache

TL;DR

This work tackles the challenge of accelerating diffusion models via caching by addressing when to cache and how to reuse caches. It introduces DiCache, a training-free, runtime adaptive approach comprised of an Online Probe Profiling Scheme (using shallow-layer signals to estimate caching error per sample) and Dynamic Cache Trajectory Alignment (combining multi-step caches via probe-informed trajectories). The method achieves higher efficiency and fidelity than state-of-the-art baselines across WAN 2.1, HunyuanVideo, and Flux, and is compatible with sparse-attention acceleration like Sparse VideoGen. The results demonstrate robust per-sample caching decisions and improved reconstruction quality, highlighting practical impact for scalable diffusion-model deployment.

Abstract

Recent years have witnessed the rapid development of acceleration techniques for diffusion models, especially caching-based acceleration methods. These studies seek to answer two fundamental questions: "When to cache" and "How to use cache", typically relying on predefined empirical laws or dataset-level priors to determine caching timings and adopting handcrafted rules for multi-step cache utilization. However, given the highly dynamic nature of the diffusion process, they often exhibit limited generalizability and fail to cope with diverse samples. In this paper, a strong sample-specific correlation is revealed between the variation patterns of the shallow-layer feature differences in the diffusion model and those of deep-layer features. Moreover, we have observed that the features from different model layers form similar trajectories. Based on these observations, we present DiCache, a novel training-free adaptive caching strategy for accelerating diffusion models at runtime, answering both when and how to cache within a unified framework. Specifically, DiCache is composed of two principal components: (1) Online Probe Profiling Scheme leverages a shallow-layer online probe to obtain an on-the-fly indicator for the caching error in real time, enabling the model to dynamically customize the caching schedule for each sample. (2) Dynamic Cache Trajectory Alignment adaptively approximates the deep-layer feature output from multi-step historical caches based on the shallow-layer feature trajectory, facilitating higher visual quality. Extensive experiments validate DiCache's capability in achieving higher efficiency and improved fidelity over state-of-the-art approaches on various leading diffusion models including WAN 2.1, HunyuanVideo and Flux.

Figures (15)

• Figure 1: Comparison between our proposed DiCache and previous caching methods. (a) Previous studies typically rely on dataset priors or empirical laws to skip timesteps, and resort to handcrafted rules to utilize multi-step caches. (b) DiCache employs an online probe to adaptively adjust its caching strategy at runtime. (c) A brief visual comparison between DiCache and existing state-of-the-art methods, in which DiCache demonstrates superiority in both quality and efficiency.
• Figure 2: Overview of DiCache. The proposed DiCache consists of Online Probe Profiling Strategy and Dynamic Cache Trajectory Alignment. The former dynamically determines the caching timing with an online shallow-layer probe at runtime, while the latter combines multi-step caches based on the probe feature trajectory to adaptively approximate the feature at the current timestep.
• Figure 3: Variation of feature differences between consecutive timesteps (Mean & Variance). (a) Output differences $\text{L1}_\text{rel}(y_t, y_{t+1})$. It exhibits large variances, indicating sample-specific nature. (b) Input differences $\text{L1}_\text{rel}(x_t, x_{t+1})$. It increases monotonically with timesteps, failing to capture the variations in output differences. (c) Shallow-layer feature differences $\text{L1}_\text{rel}(y_t^m, y_{t+1}^m)$ (5-th layer in this figure). It exhibits a strong correlation with the variations in output differences. (d) Spearman correlation coefficient between $\text{L1}_\text{rel}(y_t^m, y_{t+1}^m)$ and $\text{L1}_\text{rel}(y_t, y_{t+1})$. They already exhibit a high correlation coefficient (around 0.8) with a shallow probe depth (1$\sim$3 layers).
• Figure 4: Observation and analysis regarding Dynamic Cache Trajectory Alignment.
• Figure 5: Effects of Dynamic Cache Trajectory Alignment (DCTA). Best viewed zoomed in.