Using Powerful Prior Knowledge of Diffusion Model in Deep Unfolding Networks for Image Compressive Sensing

TL;DR

This work integrates pre-trained diffusion priors into Deep Unfolding Networks for image compressive sensing via Diffusion Message Passing (DMP). By embedding a diffusion step in each iteration and deeply unfolding it into DMP-DUN, the method leverages powerful priors while using lightweight networks to map measurements to reverse-diffusion states and to approximate divergences, achieving state-of-the-art reconstruction with as few as two steps. DMP-DUN+ further enhances performance by optionally training with an unfrozen diffusion model, boosting PSNR/SSIM at the cost of higher training complexity. Experimental results demonstrate superior reconstruction quality and lower computational cost compared with diffusion-based and traditional DUN approaches, highlighting the practical impact of combining diffusion priors with deep unfolding for efficient image CS. The approach provides a scalable, end-to-end trainable framework that can adapt to various diffusion models and CS ratios, offering a robust path toward fast, high-fidelity image reconstruction in CS applications.

Abstract

Recently, Deep Unfolding Networks (DUNs) have achieved impressive reconstruction quality in the field of image Compressive Sensing (CS) by unfolding iterative optimization algorithms into neural networks. The reconstruction quality of DUNs depends on the learned prior knowledge, so introducing stronger prior knowledge can further improve reconstruction quality. On the other hand, pre-trained diffusion models contain powerful prior knowledge and have a solid theoretical foundation and strong scalability, but it requires a large number of iterative steps to achieve reconstruction. In this paper, we propose to use the powerful prior knowledge of pre-trained diffusion model in DUNs to achieve high-quality reconstruction with less steps for image CS. Specifically, we first design an iterative optimization algorithm named Diffusion Message Passing (DMP), which embeds a pre-trained diffusion model into each iteration process of DMP. Then, we deeply unfold the DMP algorithm into a neural network named DMP-DUN. The proposed DMP-DUN can use lightweight neural networks to achieve mapping from measurement data to the intermediate steps of the reverse diffusion process and directly approximate the divergence of the diffusion model, thereby further improving reconstruction efficiency. Extensive experiments show that our proposed DMP-DUN achieves state-of-the-art performance and requires at least only 2 steps to reconstruct the image. Codes are available at https://github.com/FengodChen/DMP-DUN-CVPR2025.

Figures (6)

• Figure 1: Comparison of the average PSNR and FLOPs of our proposed DMP-DUN with other methods on Urban100Dong2018DenoisingPD, under various CS ratio of 1%, 4%, 10%, 25% and 50%.
• Figure 2: The reconstruction effects of different methods using only 2 steps under CS ratio = 10%.
• Figure 3: Compared with different type of diffusion models under the perspective of manifold hypothesis.
• Figure 4: The reconstruction process of our proposed DMP-DUN, where DMP step represents each iteration of Diffusion Message Passing (DMP) with pre-trained Diffusion Model (i.e.$p_\theta(\mathop{\mathrm{\mathbf{x}}}\nolimits_{t-1}|\mathop{\mathrm{\mathbf{r}}}\nolimits_t)$).
• Figure 5: The comparison of the subjective visual effects between our proposed DMP-DUN, DMP-DUN$^+$, and other methods at CS ratio = 4%. The best and second best results are highlighted in red and blue colors, respectively.

Theorems & Definitions (1)

• Theorem 1