Efficient Real-world Image Super-Resolution Via Adaptive Directional Gradient Convolution

TL;DR

This paper tackles real-world image super-resolution by addressing the difficulty of recovering details in regions with complex gradient arrangements under nonlinear degradation. It introduces DGConv, a convolutional unit that embeds learnable directional gradient operations (IDG, CSG, HG, VG) and aggregation (CSA) alongside a vanilla path, fused with learnable weights to preserve efficiency via equivalent parameter fusion. An Adaptive Information Interaction Block (AIIBlock) and the Directional Gradient Perceiving Network (DGPNet) are then proposed to balance texture enhancement and contrast, yielding state-of-the-art results on Real-SR benchmarks with lower computational cost. The approach demonstrates cost-effective plug-and-play improvements across existing SR models and highlights the practical impact for real-world imaging applications.

Abstract

Real-SR endeavors to produce high-resolution images with rich details while mitigating the impact of multiple degradation factors. Although existing methods have achieved impressive achievements in detail recovery, they still fall short when addressing regions with complex gradient arrangements due to the intensity-based linear weighting feature extraction manner. Moreover, the stochastic artifacts introduced by degradation cues during the imaging process in real LR increase the disorder of the overall image details, further complicating the perception of intrinsic gradient arrangement. To address these challenges, we innovatively introduce kernel-wise differential operations within the convolutional kernel and develop several learnable directional gradient convolutions. These convolutions are integrated in parallel with a novel linear weighting mechanism to form an Adaptive Directional Gradient Convolution (DGConv), which adaptively weights and fuses the basic directional gradients to improve the gradient arrangement perception capability for both regular and irregular textures. Coupled with DGConv, we further devise a novel equivalent parameter fusion method for DGConv that maintains its rich representational capabilities while keeping computational costs consistent with a single Vanilla Convolution (VConv), enabling DGConv to improve the performance of existing super-resolution networks without incurring additional computational expenses. To better leverage the superiority of DGConv, we further develop an Adaptive Information Interaction Block (AIIBlock) to adeptly balance the enhancement of texture and contrast while meticulously investigating the interdependencies, culminating in the creation of a DGPNet for Real-SR through simple stacking. Comparative results with 15 SOTA methods across three public datasets underscore the effectiveness and efficiency of our proposed approach.

Figures (8)

• Figure 1: (a) During imaging, real low-resolution (LR) images often undergo various highly nonlinear degradations, leading to inconsistent detail and contrast loss across different regions of the image. Error represents the residuals of HR and LR. (b-c) We introduce DGConv to enhance detail reconstruction and contrast restoration capabilities across different textural and flat regions while maintaining the same inference computational cost as VConv.
• Figure 2: An overview of DGConv and DGPNet. DGConv consists of i) learnable Irregular Directional Gradient convolution (IDG), ii) learnable Regular Directional Gradient convolutions: Center-Surrounding Gradient (CSG) convolution, Vertical Gradient convolution (VG), Horizontal Gradient (HG), iii) Center-Surrounding Aggregation (CSA) convolution and Vanilla Convolution (VConv). Simple yet efficient DGPNet directly stacks $N_{block}$ AIIBlocks as the backbone, uses pixel shuffle to improve the resolution of features, and uses DGConv as image-to-feature and feature-to-image layers.
• Figure 3: Illustration of our proposed Equivalent Parameters Fusion. It can merge the multiple kernels in DGConv into a single kernel to reduce the computational cost.
• Figure 4: Comparison of model complexity and performance. Our approach achieves superior performance with fewer parameters and lower FLOPs. Note that a lower LPIPS value indicates better performance. The size of the circle represents the number of parameters.
• Figure 5: Visual comparison with existing Real-SR methods. Our method achieves better detail recovery and contrast enhancement results. Please zoom in for better visualization.