Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion

TL;DR

This work addresses the problem that post-training quantization (PTQ) introduces cumulative errors in diffusion models during iterative denoising, degrading image fidelity. It develops a theoretical framework linking per-step quantization errors, cumulative error, and their propagation, deriving a closed-form solution for cumulative error under a DDIM-like sampler. Building on this, the authors propose TCEC, a timestep-aware online compensation scheme that reconstructs per-step errors via a learned channel-wise scaling (K_t) and applies a final correction term during generation, with computationally light approximations (m=1) to keep overhead minimal. Empirically, TCEC improves SOTA PTQ methods (e.g., SVDQuant) across SDXL, SDXL-Turbo, and PixArt backbones, achieving better PSNR and perceptual metrics while reducing memory footprint by ~3.5× and incurring less than 0.5% end-to-end latency, thereby enabling high-quality, low-bit quantized diffusion generation on commodity hardware.

Abstract

Diffusion models have transformed image synthesis by establishing unprecedented quality and creativity benchmarks. Nevertheless, their large-scale deployment faces challenges due to computationally intensive iterative denoising processes. Although post-training quantization (PTQ) provides an effective pathway for accelerating sampling, the iterative nature of diffusion models causes stepwise quantization errors to accumulate progressively during generation, inevitably compromising output fidelity. To address this challenge, we develop a theoretical framework that mathematically formulates error propagation in Diffusion Models (DMs), deriving per-step quantization error propagation equations and establishing the first closed-form solution for cumulative error. Building on this theoretical foundation, we propose a timestep-aware cumulative error compensation scheme. Extensive experiments on multiple image datasets demonstrate that our compensation strategy effectively mitigates error propagation, significantly enhancing existing PTQ methods. Specifically, it achieves a 1.2 PSNR improvement over SVDQuant on SDXL W4A4, while incurring only an additional $<$ 0.5\% time overhead.

Figures (4)

• Figure 1: Visualization of Error Propagation and Correction in Quantized Diffusion Models. The gray path represents the iterative denoising process of the full-precision model $\mu$, while the brown-red path represents that of the quantized model $\tilde{\mu}$. Affected by cumulative errors $\delta_t$, its output gradually deviates from $\mu$. The green path represents the denoising process after online correction of cumulative errors, with outputs better aligned with the full-precision model.
• Figure 2: Qualitative visual results comparison. Prompt1: An alien octopus floats through a protal reading a newspaper. Prompt2: A middle-aged woman of Asian descent, her dark hair streaked with silver, appears fractured and splintered, intricately embedded within a sea of broken porcelain. The porcelain glistens with splatter paint patterns in a harmonious blend of glossy and matte blues, greens, oranges, and reds, capturing her dance in a surreal juxtaposition of movement and stillness. Her skin tone, a light hue like the porcelain, adds an almost mystical quality to her form.
• Figure 3: Flux visualization of quantization errors during denoising. We compare the full-precision model (FP16) and the quantized model (W4A8) under the prompt "hummingbird flying near a flower. 4k ultra realistic ray tracing dynamic lighting" with hyperparameters num_timesteps=4 and guidance_scale=3.5. The figure illustrates three key phenomena: (1) quantization errors accumulate as denoising progresses (Step 1 $\rightarrow$ 0.75 $\rightarrow$ 0.5 $\rightarrow$ 0.25), exhibiting distinct spatial structures; (2) the errors are strongly correlated with the model outputs, particularly along object boundaries and textured regions; and (3) high-frequency components such as feather edges and flower petals amplify the discrepancies, highlighting the timestep-dependent and output-correlated nature of error propagation in quantized diffusion models.
• Figure 4: Qualitative visual results comparison. Prompt1: Luffy from ONEPIECE, handsome face, fantasy. Prompt2: The image features a woman wearing a red shirt with an icon. She appears to be posing for the camera, and her outfit includes a pair of jeans. The woman seems to be in a good mood, as she is smiling. The background of the image is blurry, focusing more on the woman and her attire. Prompt3: Bright scene, aerial view, ancient city, fantasy, gorgeous light, mirror reflection, high detail, wide angle lens.