AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring

TL;DR

A pioneering work, Adaptive Patch Ex-iting Reversible Decoder (AdaRevD), to explore their in-sufficient decoding capability by inheriting the weights of the well-trained encoder and refactor a reversible de-coder which scales up the single-decoder training to multi-decoder training while remaining GPU memory-friendly.

Abstract

Despite the recent progress in enhancing the efficacy of image deblurring, the limited decoding capability constrains the upper limit of State-Of-The-Art (SOTA) methods. This paper proposes a pioneering work, Adaptive Patch Exiting Reversible Decoder (AdaRevD), to explore their insufficient decoding capability. By inheriting the weights of the well-trained encoder, we refactor a reversible decoder which scales up the single-decoder training to multi-decoder training while remaining GPU memory-friendly. Meanwhile, we show that our reversible structure gradually disentangles high-level degradation degree and low-level blur pattern (residual of the blur image and its sharp counterpart) from compact degradation representation. Besides, due to the spatially-variant motion blur kernels, different blur patches have various deblurring difficulties. We further introduce a classifier to learn the degradation degree of image patches, enabling them to exit at different sub-decoders for speedup. Experiments show that our AdaRevD pushes the limit of image deblurring, e.g., achieving 34.60 dB in PSNR on GoPro dataset.

Figures (14)

• Figure 1: The ranked PSNR curve of the image patches from GoPro Nah2017deep train set and the visualization of the patches with various degradation degrees (e.g. hard, moderate and simple).
• Figure 2: Visual comparison of the outputs from different sub-decoders. The first column is the difference between blur image and the first sub-decoder's output. The rest of the columns are the residual between the current sub-decoder and the former one.
• Figure 3: Architecture of AdaRevD. AdaRevD consists of three parts: a pre-trained encoder, several sub-decoders and a classifier. To push the limit of image deblurring networks, and map the learned degradation representation to the blur pattern more effectively, the pre-trained encoder is fixed during training. Each sub-decoder is composed of four Level modules, including a Fuseblock and a FourierBlock. SCA means Simple Channel Attention proposed in NAFNet Chen2022simple. The classifier predicts the degradation degree of each image patch, which allows the network to exit in the appropriate sub-decoder.
• Figure 4: The increment for degraded patches belonging to each degradation degree class in different sub-decoders. The patches are generated from GoPro Nah2017deep train set. d$(i)$-d$(i-1)$ means the average increment PSNR for the patches in the $i$th sub-decoder.
• Figure 5: Examples on the GoPro test dataset. The first row shows blur image, predicted images of different methods, and ground-truth sharp image. The second row shows the residual of the blur image / predicted sharp images and the ground-truth sharp image.