Improving Decoupled Posterior Sampling for Inverse Problems using Data Consistency Constraint

TL;DR

This work tackles ill-posed inverse problems solved via diffusion-based posterior sampling, where early-step bias arises when measurement information is underutilized. The authors introduce Guided Decoupled Posterior Sampling (GDPS), which imposes a data-consistency constraint in the reverse diffusion path to steer the optimization toward the true posterior, and extend the approach to latent diffusion models and Tweedie’s formula. Empirical results on FFHQ and ImageNet across a range of linear and nonlinear tasks show that GDPS yields the best reported performance among Decoupled Posterior Sampling methods, including variants like LatentDAPS and SITCOM. The method demonstrates strong generalization and scalability, improving reconstruction quality while maintaining competitive running time on standard hardware, thereby offering a practical and effective improvement for a broad class of inverse problems.

Abstract

Diffusion models have shown strong performances in solving inverse problems through posterior sampling while they suffer from errors during earlier steps. To mitigate this issue, several Decoupled Posterior Sampling methods have been recently proposed. However, the reverse process in these methods ignores measurement information, leading to errors that impede effective optimization in subsequent steps. To solve this problem, we propose Guided Decoupled Posterior Sampling (GDPS) by integrating a data consistency constraint in the reverse process. The constraint performs a smoother transition within the optimization process, facilitating a more effective convergence toward the target distribution. Furthermore, we extend our method to latent diffusion models and Tweedie's formula, demonstrating its scalability. We evaluate GDPS on the FFHQ and ImageNet datasets across various linear and nonlinear tasks under both standard and challenging conditions. Experimental results demonstrate that GDPS achieves state-of-the-art performance, improving accuracy over existing methods.

Figures (11)

• Figure 1: Representative Results for the DPS, DAPS, and GDPS Methods. The tasks are presented in the following order: (a) super resolution, (b) Gaussian deblurring, (c) nonlinear deblurring, and (d) phase retrieval. As shown in Figure 1, our GDPS method consistently achieves higher quality results compared to other methods. In Figure 1(a), our method successfully reconstructs clear wrinkles from the measurements, enhancing facial texture. In Figure 1(b), GDPS accurately recovers fine details around the eye area, including the eye bags and eye socket, demonstrating improved facial feature restoration. In Figure 1(c), our method effectively restores detailed elements such as the eye shadow from the measurements. Finally, in Figure 1(d), GDPS reconstructs intricate details of the headwear and background with higher clarity. Overall, the visual quality of GDPS’s results is closer to the reference images, underscoring its effectiveness in enhancing clarity and preserving fine details.
• Figure 2: Illustrative Diagram of Our Method GDPS. As shown in the figure, we begin by sending the input $\mathbf{x}_t$ into the guided reverse process. In this process, we introduce a guidance term to optimize $\mathbf{x}_{t_k}$ into $\mathbf{x}_{t_k}^{\text{guided}}$, progressively moving towards a clear sample, $\hat{\mathbf{x}}_0(\mathbf{x}_t)$. In the optimization process, we apply Langevin dynamics to sample $\mathbf{x}_{0|\mathbf{y}}$ from the posterior distribution $p(\mathbf{x}_0 \mid \mathbf{x}_t, \mathbf{y})$ based on $\hat{\mathbf{x}}_0(\mathbf{x}_t)$. Finally, in the forward process, noise is reintroduced into the sample to obtain the next noisy sample $\mathbf{x}_{t-1}$, continuing the iterative procedure.
• Figure 3: Representative results from the experiments in the validation set of FFHQ 256x256 dataset. Arranged from top to bottom, the taks are: super resolution, gaussian deblurring, and nonlinear deblurring. To highlight areas with quality improvements, we have marked and enlarged specific regions within each image.
• Figure 4: Representative results from the experiments in the validation set of ImageNet 256x256 dataset. Arranged from top to bottom, the taks are: super resolution, gaussian deblurring, and nonlinear deblurring. To highlight areas with quality improvements, we have marked and enlarged specific regions within each image.
• Figure 5: Representative results from the challenging experiments in the validation set of FFHQ 256x256 dataset. Arranged from top to bottom, the taks are: super resolution(16x), box inpainting(192x192), and inpainting(90%). To highlight areas with quality improvements, we have marked and enlarged specific regions within each image.