A Preliminary Exploration Towards General Image Restoration

TL;DR

This paper delineates the essential aspects of GIR, including problem definition and the overarching significance of generalization performance, and conducts a comprehensive evaluation of existing approaches for tackling the GIR challenge, illuminating their strengths and pragmatic challenges.

Abstract

Despite the tremendous success of deep models in various individual image restoration tasks, there are at least two major technical challenges preventing these works from being applied to real-world usages: (1) the lack of generalization ability and (2) the complex and unknown degradations in real-world scenarios. Existing deep models, tailored for specific individual image restoration tasks, often fall short in effectively addressing these challenges. In this paper, we present a new problem called general image restoration (GIR) which aims to address these challenges within a unified model. GIR covers most individual image restoration tasks (\eg, image denoising, deblurring, deraining and super-resolution) and their combinations for general purposes. This paper proceeds to delineate the essential aspects of GIR, including problem definition and the overarching significance of generalization performance. Moreover, the establishment of new datasets and a thorough evaluation framework for GIR models is discussed. We conduct a comprehensive evaluation of existing approaches for tackling the GIR challenge, illuminating their strengths and pragmatic challenges. By analyzing these approaches, we not only underscore the effectiveness of GIR but also highlight the difficulties in its practical implementation. At last, we also try to understand and interpret these models' behaviors to inspire the future direction. Our work can open up new valuable research directions and contribute to the research of general vision.

Figures (11)

• Figure 1: The differences between the General Image Restoration (GIR) and the General High-level Vision (GHV).
• Figure 2: The Rain1200 result of models with different ability. The resulting image of derain&djpeg model obtains lower PSNR value, even though it looks more natural and cleaner. This issue arises from the reference GT images lacking the desired level of pristine quality and idealized perfection.
• Figure 3: The demo of ground truth images. They contain 10 scenes with 10 images per scene from Unsplash unsplash.
• Figure 4: The demo of real world images. They contain 10 types with 100 real-world testing images per type.
• Figure 5: Benchmark results of the existing approaches using the GIR evaluation protocol. (a) shows the scatter plot of models performance on all synthetic testing tasks; (b) shows the task bias of Uformer and (c) shows the task bias of Restormer, please refer to Section \ref{['sec: Practical Difficulties']} for more details; (d) shows the performance of MIR-10, MIR-50 and GIR models on different degradations respectively.