Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey

TL;DR

This survey tackles the problem of restoring and enhancing degraded images using diffusion models, detailing two dominant workflows: supervised DM-based IR and zero-shot DM-based IR, including extensions for blind/real-world degradations. It provides a systematic taxonomy by conditioning strategies and generation spaces, surveys representative methods (e.g., SR3, ILVR) and their variants, and compiles datasets, metrics, and cross-task performance across SR, deblurring, and inpainting. The paper also discusses practical challenges—sampling efficiency, model size, distortion handling—and outlines nine future directions, including all-in-one restoration, LLM/MLLM-driven agents, and vision-language prompting, to guide research and deployment. Overall, it offers a foundational, task-oriented framework for understanding and advancing diffusion-based image restoration in both academic and applied settings.

Abstract

Image restoration (IR) has been an indispensable and challenging task in the low-level vision field, which strives to improve the subjective quality of images distorted by various forms of degradation. Recently, the diffusion model has achieved significant advancements in the visual generation of AIGC, thereby raising an intuitive question, "whether diffusion model can boost image restoration". To answer this, some pioneering studies attempt to integrate diffusion models into the image restoration task, resulting in superior performances than previous GAN-based methods. Despite that, a comprehensive and enlightening survey on diffusion model-based image restoration remains scarce. In this paper, we are the first to present a comprehensive review of recent diffusion model-based methods on image restoration, encompassing the learning paradigm, conditional strategy, framework design, modeling strategy, and evaluation. Concretely, we first introduce the background of the diffusion model briefly and then present two prevalent workflows that exploit diffusion models in image restoration. Subsequently, we classify and emphasize the innovative designs using diffusion models for both IR and blind/real-world IR, intending to inspire future development. To evaluate existing methods thoroughly, we summarize the commonly-used dataset, implementation details, and evaluation metrics. Additionally, we present the objective comparison for open-sourced methods across three tasks, including image super-resolution, deblurring, and inpainting. Ultimately, informed by the limitations in existing works, we propose five potential and challenging directions for the future research of diffusion model-based IR, including sampling efficiency, model compression, distortion simulation and estimation, distortion invariant learning, and framework design.

Figures (19)

• Figure 1: Denoising Diffusion Probabilistic Models.
• Figure 2: The overview of diffusion model-based image restoration models. This figure categorizes diffusion models into two types based on their training methods, namely the supervised-based models (indicated with a blue background) and zero-shot-based models (indicated with an orange background). Additionally, the figure provides a more detailed classification of models within these two categories according to how conditions are incorporated. The models with a green background are specifically designed for real-world image restoration. All works in this figure can be found in Section C, Table 1 and Table 2 of the Supplementary).
• Figure 3: The Backbone of SR3 network
• Figure 4: The Backbone of ILVR network
• Figure 5: The flowchart of DM-based IR methods using pre-processed references, where the low-quality image is first processed by a pre-trained restoration network or trainable preprocessing module. The output references could be features or clean images