Towards Context-aware Convolutional Network for Image Restoration

TL;DR

This paper tackles image restoration under dynamic, non-uniform degradations by proposing a context-aware CNN (CCNet) that leverages an Efficient Residual Star Module (ERSM) and a Large Dynamic Integration Module (LDIM) to achieve long-range contextual modeling with low complexity. ERSM maps features into high-dimensional nonlinear spaces via a context-aware star operation, while LDIM expands the receptive field to dynamically fuse diverse contextual cues. The network uses a six-scale encoder–decoder with skip connections and residual learning to reinforce training. Experiments on image dehazing, motion deblurring, and desnowing show CCNet delivers state-of-the-art performance with lower parameter counts and MACs compared to transformer-based rivals.

Abstract

Image restoration (IR) is a long-standing task to recover a high-quality image from its corrupted observation. Recently, transformer-based algorithms and some attention-based convolutional neural networks (CNNs) have presented promising results on several IR tasks. However, existing convolutional residual building modules for IR encounter limited ability to map inputs into high-dimensional and non-linear feature spaces, and their local receptive fields have difficulty in capturing long-range context information like Transformer. Besides, CNN-based attention modules for IR either face static abundant parameters or have limited receptive fields. To address the first issue, we propose an efficient residual star module (ERSM) that includes context-aware "star operation" (element-wise multiplication) to contextually map features into exceedingly high-dimensional and non-linear feature spaces, which greatly enhances representation learning. To further boost the extraction of contextual information, as for the second issue, we propose a large dynamic integration module (LDIM) which possesses an extremely large receptive field. Thus, LDIM can dynamically and efficiently integrate more contextual information that helps to further significantly improve the reconstruction performance. Integrating ERSM and LDIM into an U-shaped backbone, we propose a context-aware convolutional network (CCNet) with powerful learning ability for contextual high-dimensional mapping and abundant contextual information. Extensive experiments show that our CCNet with low model complexity achieves superior performance compared to other state-of-the-art IR methods on several IR tasks, including image dehazing, image motion deblurring, and image desnowing.

Figures (11)

• Figure 1: Performance and parameters of different methods on SOTS-Outdoor dataset.
• Figure 2: Network architecture of our CCNet.
• Figure 3: The details of different modules. (a) Lightweight star block (StarB) in ref43 for high-level tasks. (b) The proposed efficient residual star module (ERSM) which includes context-aware star operation for low-level tasks. (c) The designed D-RSM for comparison.
• Figure 4: The details of the proposed horizontal large dynamic strip integration (H-LDSI), vertical large dynamic strip integration (V-LDSI), and large dynamic integration module (LDIM).
• Figure 5: The details of the proposed context-aware residual star attention module (RSAM).