FAST-DIPS: Adjoint-Free Analytic Steps and Hard-Constrained Likelihood Correction for Diffusion-Prior Inverse Problems

TL;DR

This work proves local model optimality and descent under backtracking for the step-size rule, and derive an explicit KL bound for mode-substitution re-annealing under a local Gaussian conditional surrogate.

Abstract

Training-free diffusion priors enable inverse-problem solvers without retraining, but for nonlinear forward operators data consistency often relies on repeated derivatives or inner optimization/MCMC loops with conservative step sizes, incurring many iterations and denoiser/score evaluations. We propose a training-free solver that replaces these inner loops with a hard measurement-space feasibility constraint (closed-form projection) and an analytic, model-optimal step size, enabling a small, fixed compute budget per noise level. Anchored at the denoiser prediction, the correction is approximated via an adjoint-free, ADMM-style splitting with projection and a few steepest-descent updates, using one VJP and either one JVP or a forward-difference probe, followed by backtracking and decoupled re-annealing. We prove local model optimality and descent under backtracking for the step-size rule, and derive an explicit KL bound for mode-substitution re-annealing under a local Gaussian conditional surrogate. We also develop a latent variant and a one-parameter pixel$\rightarrow$latent hybrid schedule. Experiments achieve competitive PSNR/SSIM/LPIPS with up to 19.5$\times$ speedup, without hand-coded adjoints or inner MCMC.

Figures (23)

• Figure 1: FFHQ results on four inverse problems: (a) Gaussian deblurring, (b) phase retrieval, (c) random inpainting, (d) HDR. Each panel shows the measurement, baselines, our fast-dips output, and the reference. SSIM/LPIPS and average per-image run-time (s) are overlaid; fast-dips attains comparable or higher quality with markedly lower run-time.
• Figure 2: fast-dips method sketch. At each noise level $t$, we (1) compute a denoiser anchor $\mathbf{x}_{0\mid t}$, (2) perform a hard-constrained measurement-space correction under an $\ell_2$ feasibility ball via a few-step ADMM-style splitting (closed-form projection + a few descent steps with analytic $\alpha^\star$ using one VJP and either one JVP or a forward-difference probe), and (3) re-anneal to obtain $\mathbf{x}_{t-1}$.
• Figure 3: Quantitative evaluations comparing image quality and computational time for baseline methods. Each point is derived from an experiment on 100 FFHQ images. The y-axis value (PSNR or LPIPS) is the mean of the scores from the 100 resulting images. The x-axis value is the average per-image run-time, calculated by dividing the total processing time for all 100 images by 100. The plots show results for three linear tasks (a-c) and one nonlinear task (d).
• Figure 4: Quantitative results under Poisson measurement noise ($\lambda_\text{poisson}=1$). FAST-DIPS remains accurate and perceptually faithful across tasks.
• Figure 5: Qualitative results for JPEG restoration on FFHQ using FAST-DIPS with a differentiable surrogate operator. We display the measurement, reconstruction, and the ground-truth reference across three compression levels: JPEG Quality 5, 10, and 25.

Theorems & Definitions (20)

• Proposition 1: Local model-optimal step and descent
• Remark 1: Nonconvexity and scope
• Proposition 2: Latent model-optimal step and descent
• Proposition 3: KKT at latent ADMM fixed points
• Remark 2: Transfer of pixel-space results
• Remark 3: Consistency of pixel $\to$ encode with latent correction
• Proposition 4: Closed-form projection onto the measurement ball
• proof : Proof of Proposition \ref{['prop:projection-ball']}
• Proposition 4: Local model-optimal step and descent
• proof : Proof of Proposition \ref{['prop:alphastar-method']}