Steering One-Step Diffusion Model with Fidelity-Rich Decoder for Fast Image Compression

TL;DR

This paper tackles the latency and fidelity challenges of diffusion-based image compression by introducing SODEC, a single-step diffusion framework guided by a fidelity-rich decoder. It combines a VAE-based compression backbone with a one-step diffusion process and a fidelity guidance module, trained via a three-stage rate annealing schedule. Empirical results show state-of-the-art rate-distortion-perception performance and over 20× decoding speedups compared to multi-step diffusion methods. The approach yields high-fidelity reconstructions at ultra-low bitrates while maintaining perceptual realism, making diffusion-based compression more practical for real-time or resource-constrained applications.

Abstract

Diffusion-based image compression has demonstrated impressive perceptual performance. However, it suffers from two critical drawbacks: (1) excessive decoding latency due to multi-step sampling, and (2) poor fidelity resulting from over-reliance on generative priors. To address these issues, we propose SODEC, a novel single-step diffusion image compression model. We argue that in image compression, a sufficiently informative latent renders multi-step refinement unnecessary. Based on this insight, we leverage a pre-trained VAE-based model to produce latents with rich information, and replace the iterative denoising process with a single-step decoding. Meanwhile, to improve fidelity, we introduce the fidelity guidance module, encouraging output that is faithful to the original image. Furthermore, we design the rate annealing training strategy to enable effective training under extremely low bitrates. Extensive experiments show that SODEC significantly outperforms existing methods, achieving superior rate-distortion-perception performance. Moreover, compared to previous diffusion-based compression models, SODEC improves decoding speed by more than 20$\times$. Code is released at: https://github.com/zhengchen1999/SODEC.

Figures (5)

• Figure 1: LPIPS-bitrate-latency comparison on DIV2K-Val. Decoding time is measured on 512$\times$512 images using one A6000 GPU. Our method achieves the best perceptual quality (i.e., LPIPS). Meanwhile, compared to the multi-step diffusion-based method DiffEIC li2024towards, our method offers a 38$\times$ speedup in decoding time.
• Figure 2: Overview of SODEC. (a) VAE compression module: A pre-trained VAE-based compression model is used to generate the informative latent representation. (b) One-step diffusion model: The latent is mapped to the diffusion space via the transformation module, followed by single-step denoising to produce the reconstructed output. (c) Fidelity guidance module (FGM): A high-fidelity preliminary reconstruction is generated using the VAE-based compression model. Then, the pre-trained ViT is used to extract visual features as the guidance for the diffusion model.
• Figure 3: Fidelity comparison (i.e., MS-SSIM) on DIV2K-Val. We compare MS-SSIM (with GT) under different bitrates for the fidelity reconstruction and the diffusion outputs with (w/) and without (w/o) the fidelity guidance module (FGM). The use of FGM improves reconstruction fidelity.
• Figure 4: Quantitative comparison with state-of-the-art methods on the Kodak, DIV2K-Val, and CLIC2020 datasets.
• Figure 5: Qualitative comparison with state-of-the-art methods on the Kodak, DIV2K-Val, and CLIC2020 datasets.