Diffusion models for inverse problems
Hyungjin Chung, Jeongsol Kim, Jong Chul Ye
TL;DR
Diffusion priors offer a flexible framework for solving ill-posed imaging inverse problems by turning priors into trainable score functions. The chapter surveys score-based and variational viewpoints and partitions DIS into explicit approximation methods, variational/inference approaches, decoupled data-consistency, and sequential Monte Carlo, then extends these solvers to blind problems, 3D reconstructions, data-scarcity scenarios, and multimodal text conditioning. It highlights core ideas such as PF-ODE, DDIM, Jensen’s approximation, Tweedie’s formula, and various manifold- and flow-based extensions, clarifying trade-offs between speed, accuracy, and robustness. The discussion points toward future directions in reconciling speed-accuracy, scalability, and multimodal conditioning for broad, practical impact in imaging.
Abstract
Using diffusion priors to solve inverse problems in imaging have significantly matured over the years. In this chapter, we review the various different approaches that were proposed over the years. We categorize the approaches into the more classic explicit approximation approaches and others, which include variational inference, sequential monte carlo, and decoupled data consistency. We cover the extension to more challenging situations, including blind cases, high-dimensional data, and problems under data scarcity and distribution mismatch. More recent approaches that aim to leverage multimodal information through texts are covered. Through this chapter, we aim to (i) distill the common mathematical threads that connect these algorithms, (ii) systematically contrast their assumptions and performance trade-offs across representative inverse problems, and (iii) spotlight the open theoretical and practical challenges by clarifying the landscape of diffusion model based inverse problem solvers.