Suppressing Uncertainties in Degradation Estimation for Blind Super-Resolution

TL;DR

By suppressing the uncertainties of local degradation representations in images, USR facilitated self-supervised learning of degradation representations, and an Uncertainty-based degradation representation for blind Super-Resolution framework (USR) is proposed.

Abstract

The problem of blind image super-resolution aims to recover high-resolution (HR) images from low-resolution (LR) images with unknown degradation modes. Most existing methods model the image degradation process using blur kernels. However, this explicit modeling approach struggles to cover the complex and varied degradation processes encountered in the real world, such as high-order combinations of JPEG compression, blur, and noise. Implicit modeling for the degradation process can effectively overcome this issue, but a key challenge of implicit modeling is the lack of accurate ground truth labels for the degradation process to conduct supervised training. To overcome this limitations inherent in implicit modeling, we propose an \textbf{U}ncertainty-based degradation representation for blind \textbf{S}uper-\textbf{R}esolution framework (\textbf{USR}). By suppressing the uncertainty of local degradation representations in images, USR facilitated self-supervised learning of degradation representations. The USR consists of two components: Adaptive Uncertainty-Aware Degradation Extraction (AUDE) and a feature extraction network composed of Variable Depth Dynamic Convolution (VDDC) blocks. To extract Uncertainty-based Degradation Representation from LR images, the AUDE utilizes the Self-supervised Uncertainty Contrast module with Uncertainty Suppression Loss to suppress the inherent model uncertainty of the Degradation Extractor. Furthermore, VDDC block integrates degradation information through dynamic convolution. Rhe VDDC also employs an Adaptive Intensity Scaling operation that adaptively adjusts the degradation representation according to the network hierarchy, thereby facilitating the effective integration of degradation information. Quantitative and qualitative experiments affirm the superiority of our approach.

Figures (5)

• Figure 1: Comparison on degradation estimation stability. We randomly select different patches from the same image to compare the mean and variance of the degradation representations obtained by DASR, KDSR and USR. These methods exhibit varying degrees of instability, rooted in the inherent uncertainties of the model. USR (ours) demonstrates the most stable performance among them.
• Figure 2: The proposed framework Uncertainty-based degradation representation for blind Super-Resolution (USR). (a) Illustration of the main process of USR. USR extracts Uncertainty-based Degradation Representation (UDR) from the LR image, which is then integrated with the super-resolution process through the VDDC. Reconstruction refers to the process of upsampling features. (b) Depiction of the Adaptive Uncertainty-Aware Degradation Extraction (AUDE). AUDE trains the Degradation Extractor (DE) in a self-supervised manner. (c) Depiction of the Variable Depth Dynamic Convolution (VDDC) Block. VDDC integrates UDR while extracting deep features from the LR image.
• Figure 3: Visual comparisons of several representative methods on examples of the DIV2K dataset. The image above is a case of stained glass, while the image below depicts a urban street scene.
• Figure 4: Visualization of ablation study on USLoss. $L_U$ and $L_{UR}$ represent the two components of USLoss. (a) represents USR trained without $L_U$; (b) depicts USR trained without $L_{UR}$; (c) shows USR trained with USLoss.
• Figure 5: The t-SNE visualizations on the DIV2K datasets. Blur, noise, and JPEG compression represent common degradation modes in real-world scenarios. We conducted cluster analysis experiments under their various combinations. USR (Ours) effectively distinguishes between different degradation modes.