Beyond Few-Step Inference: Accelerating Video Diffusion Transformer Model Serving with Inter-Request Caching Reuse

Abstract

Video Diffusion Transformer (DiT) models are a dominant approach for high-quality video generation but suffer from high inference cost due to iterative denoising. Existing caching approaches primarily exploit similarity within the diffusion process of a single request to skip redundant denoising steps. In this paper, we introduce Chorus, a caching approach that leverages similarity across requests to accelerate video diffusion model serving. Chorus achieves up to 45\% speedup on industrial 4-step distilled models, where prior intra-request caching approaches are ineffective. Particularly, Chorus employs a three-stage caching strategy along the denoising process. Stage 1 performs full reuse of latent features from similar requests. Stage 2 exploits inter-request caching in specific latent regions during intermediate denoising steps. This stage is combined with Token-Guided Attention Amplification to improve semantic alignment between the generated video and the conditional prompts, thereby extending the applicability of full reuse to later denoising steps.

Figures (12)

• Figure 1: The video generation illustration of Chorus on a distilled 4-step Wan2.1 model. Chorus reuses cached latent states from a semantically similar request. It significantly accelerates inference while maintaining accurate semantic generation.
• Figure 2: The DiT model architecture and inference process.
• Figure 3: Illustration of intra- and inter-request caching reuse, with denoising timesteps explicitly expanded.
• Figure 4: Chorus Overview.
• Figure 5: The illustration of key amplification in TGAA.We dynamically amplify key vectors of condition tokens (e.g., “African”, “wild”) that best capture the semantic difference in cross-attention operator to guide video latents.