Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CorGi: Contribution-Guided Block-Wise Interval Caching for Training-Free Acceleration of Diffusion Transformers

Yonglak Son, Suhyeok Kim, Seungryong Kim, Young Geun Kim

TL;DR

CorGi introduces a training-free approach to accelerate Diffusion Transformer inference by identifying low-contribution transformer blocks via a CK A-based contribution score and caching them across denoising intervals. It combines warm-up followed by interval caching with an intra-block strategy to preserve information flow, and extends to CorGi+ which protects salient tokens in text-to-image generation using per-block cross-attention. The method yields up to roughly 2× end-to-end speedups across multiple DiT models (SD3.5 Large, FLUX, PixArt-Sigma) while maintaining or improving generation quality, outperforming existing block-segment and token-wise caching baselines. This work enables practical deployment of large diffusion models with minimal retraining, offering a controllable balance between speed and fidelity grounded in block-level contributions and attentive token protection.

Abstract

Diffusion transformer (DiT) achieves remarkable performance in visual generation, but its iterative denoising process combined with larger capacity leads to a high inference cost. Recent works have demonstrated that the iterative denoising process of DiT models involves substantial redundant computation across steps. To effectively reduce the redundant computation in DiT, we propose CorGi (Contribution-Guided Block-Wise Interval Caching), training-free DiT inference acceleration framework that selectively reuses the outputs of transformer blocks in DiT across denoising steps. CorGi caches low-contribution blocks and reuses them in later steps within each interval to reduce redundant computation while preserving generation quality. For text-to-image tasks, we further propose CorGi+, which leverages per-block cross-attention maps to identify salient tokens and applies partial attention updates to protect important object details. Evaluation on the state-of-the-art DiT models demonstrates that CorGi and CorGi+ achieve up to 2.0x speedup on average, while preserving high generation quality.

CorGi: Contribution-Guided Block-Wise Interval Caching for Training-Free Acceleration of Diffusion Transformers

TL;DR

CorGi introduces a training-free approach to accelerate Diffusion Transformer inference by identifying low-contribution transformer blocks via a CK A-based contribution score and caching them across denoising intervals. It combines warm-up followed by interval caching with an intra-block strategy to preserve information flow, and extends to CorGi+ which protects salient tokens in text-to-image generation using per-block cross-attention. The method yields up to roughly 2× end-to-end speedups across multiple DiT models (SD3.5 Large, FLUX, PixArt-Sigma) while maintaining or improving generation quality, outperforming existing block-segment and token-wise caching baselines. This work enables practical deployment of large diffusion models with minimal retraining, offering a controllable balance between speed and fidelity grounded in block-level contributions and attentive token protection.

Abstract

Diffusion transformer (DiT) achieves remarkable performance in visual generation, but its iterative denoising process combined with larger capacity leads to a high inference cost. Recent works have demonstrated that the iterative denoising process of DiT models involves substantial redundant computation across steps. To effectively reduce the redundant computation in DiT, we propose CorGi (Contribution-Guided Block-Wise Interval Caching), training-free DiT inference acceleration framework that selectively reuses the outputs of transformer blocks in DiT across denoising steps. CorGi caches low-contribution blocks and reuses them in later steps within each interval to reduce redundant computation while preserving generation quality. For text-to-image tasks, we further propose CorGi+, which leverages per-block cross-attention maps to identify salient tokens and applies partial attention updates to protect important object details. Evaluation on the state-of-the-art DiT models demonstrates that CorGi and CorGi+ achieve up to 2.0x speedup on average, while preserving high generation quality.
Paper Structure (42 sections, 9 equations, 20 figures, 4 tables)

This paper contains 42 sections, 9 equations, 20 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Diffusion Models
  4. Forward and Reverse Process of Diffusion:
  5. Diffusion Transformers:
  6. Redundant Computation in DiT
  7. Feature Caching Methods for DiT
  8. Block-Segment Caching:
  9. Token-Wise Caching:
  10. Methodology
  11. Block-Wise Contribution Analysis
  12. Caching Interval:
  13. Block-Wise Caching
  14. Intra-Block Caching Strategy:
  15. CorGi: Contribution-Guided Block-Wise Interval Caching
  16. ...and 27 more sections

Figures (20)

  • Figure 1: Teaser. Our framework accelerates diffusion transformers (DiT) by 2.02$\times$ on (a) Stable Diffusion 3.5-Large and by 1.91$\times$ on (b) FLUX.1-dev, while preserving high image quality and consistency with images generated by the original model.
  • Figure 2: Similarity and redundancy across adjacent denoising steps. CKA between the predicted noises of the adjacent denoising steps of (a) SD3.5 and (b) FLUX.1. The representations show only marginal difference across steps, as highlighted in gray-shaded area. Note the colored area represents the range across different prompts, and the colored line represents their mean value.
  • Figure 3: Overview of feature caching methods for DiT. (a) Block-segment caching, which reuses previous step's block segment. (b) Token-wise caching, which computes salient tokens and merges them with cached output.
  • Figure 4: Block-wise contribution analysis in denoising process. (a) Token-wise cosine similarity between the outputs from the original model and the block-pruned model of SD3.5. (b) CKA on the same pairs, evaluated across all blocks and steps
  • Figure 5: Overview of block-wise caching. (a) Naive block-wise caching which caches low-contributed blocks at each step for reuse. (b) Block-wise interval caching which refreshes cached features through full computation at intervals.
  • ...and 15 more figures