Toward Efficient Deep Blind RAW Image Restoration

TL;DR

A new realistic degradation pipeline for training deep blind RAW restoration models that considers realistic sensor noise, motion blur, camera shake, and other common degradations and is considered to be the most exhaustive analysis on RAW image restoration.

Abstract

Multiple low-vision tasks such as denoising, deblurring and super-resolution depart from RGB images and further reduce the degradations, improving the quality. However, modeling the degradations in the sRGB domain is complicated because of the Image Signal Processor (ISP) transformations. Despite of this known issue, very few methods in the literature work directly with sensor RAW images. In this work we tackle image restoration directly in the RAW domain. We design a new realistic degradation pipeline for training deep blind RAW restoration models. Our pipeline considers realistic sensor noise, motion blur, camera shake, and other common degradations. The models trained with our pipeline and data from multiple sensors, can successfully reduce noise and blur, and recover details in RAW images captured from different cameras. To the best of our knowledge, this is the most exhaustive analysis on RAW image restoration. Code available at https://github.com/mv-lab/AISP

Figures (6)

• Figure 1: Comparison of image restoration pipelines. Note that previous approaches (a) depend on the in-camera ISP output, meanwhile, our RawIR method can complement any (learned) ISP, as the RAW restoration happens before it. We refer as RAW* and RGB* to the enhanced images.
• Figure 2: Degraded samples from our datasets. Five real, and five generated. To avoid biases, we do not indicate if the images are real or synthetic. We can appreciate defocus, motion blur, and noise among the most common degradations. The synthetic training dataset covers these degradations thanks to our degradation pipeline. Best viewed in electronic version.
• Figure 3: Samples generated using our RAW degradation pipeline. For a reference input RAW image, a degradation sequence is performed to produce the corresponding degraded image (see Sec. \ref{['ssc:summary']}). Thus, we can generate paired degraded/clean images for training deep blind restoration model. Each degradation is applied on the 4-channel RAW image liu2019learningrawaug respecting the RGGB pattern. RAW images are visualized through bilinear demosaicing. Image from our dataset.
• Figure 4: Illustration of RawIR, based on NAFNetchen2022simple. We incorporate our designed Dynamic Convolution Block (DynConvT) verelst2020dynamic into the decoder, and reduce the number of blocks. More details in the appendix.
• Figure 5: Qualitative results of RAW restoration on a real-world capture (defocus and high ISO sample).