LIR: A Lightweight Baseline for Image Restoration

TL;DR

A Lightweight Baseline network for Image Restoration called LIR is proposed to efficiently restore the image and remove degradations and produces better visual results that are more in line with the human aesthetic.

Abstract

Recently, there have been significant advancements in Image Restoration based on CNN and transformer. However, the inherent characteristics of the Image Restoration task are often overlooked in many works. They, instead, tend to focus on the basic block design and stack numerous such blocks to the model, leading to parameters redundant and computations unnecessary. Thus, the efficiency of the image restoration is hindered. In this paper, we propose a Lightweight Baseline network for Image Restoration called LIR to efficiently restore the image and remove degradations. First of all, through an ingenious structural design, LIR removes the degradations existing in the local and global residual connections that are ignored by modern networks. Then, a Lightweight Adaptive Attention (LAA) Block is introduced which is mainly composed of proposed Adaptive Filters and Attention Blocks. The proposed Adaptive Filter is used to adaptively extract high-frequency information and enhance object contours in various IR tasks, and Attention Block involves a novel Patch Attention module to approximate the self-attention part of the transformer. On the deraining task, our LIR achieves the state-of-the-art Structure Similarity Index Measure (SSIM) and comparable performance to state-of-the-art models on Peak Signal-to-Noise Ratio (PSNR). For denoising, dehazing, and deblurring tasks, LIR also achieves a comparable performance to state-of-the-art models with a parameter size of about 30\%. In addition, it is worth noting that our LIR produces better visual results that are more in line with the human aesthetic.

Figures (9)

• Figure 1: Visualization experiment on the deraining task. (a) The paradigm of the most modern networks that incorporate both local and global residual connections. (b) The restored image output by the state-of-the-art model, Restormer 34zamir2022restormer. It still exhibits visible traces of rain. (c) The restored image output by our LIR. It is cleaner and solves the problem of residual connections carrying degradation. (d) The visualization of intermediate feature maps of PReNet 55ren2019progressive, SFNet 51cui2022selective, IRNeXt cui2023irnext, Restormer, and LIR.
• Figure 2: The detailed structure of the proposed Lightweight Image Restoration (LIR) network, which is mainly stacked by Lightweight Adaptive Attention (LAA) Blocks. Thanks to the Adaptive Filter, Patch Attention, ResCABlock, and the ResBlock, the LAA Block is endowed with the ability to extract a variety of information efficiently.
• Figure 3: The detailed architecture of the proposed Adaptive Filter. It is a reparameterization style that multiple convolutions are reparameterized into one during inference after the training.
• Figure 4: The flow of the proposed Patch Attention.
• Figure 5: The visual comparisons for LIR, Restormer 34zamir2022restormer, SFNet 51cui2022selective, and HINet 50chen2021hinet. LIR produces state-of-the-art visual results.