Iterative Network for Image Super-Resolution

TL;DR

A novel iterative super-resolution network (ISRN) is proposed on top of the iterative optimization of conventional SISR algorithm, and a feature normalization (F-Norm, FN) method to regulate the features in network is proposed.

Abstract

Single image super-resolution (SISR), as a traditional ill-conditioned inverse problem, has been greatly revitalized by the recent development of convolutional neural networks (CNN). These CNN-based methods generally map a low-resolution image to its corresponding high-resolution version with sophisticated network structures and loss functions, showing impressive performances. This paper provides a new insight on conventional SISR algorithm, and proposes a substantially different approach relying on the iterative optimization. A novel iterative super-resolution network (ISRN) is proposed on top of the iterative optimization. We first analyze the observation model of image SR problem, inspiring a feasible solution by mimicking and fusing each iteration in a more general and efficient manner. Considering the drawbacks of batch normalization, we propose a feature normalization (F-Norm, FN) method to regulate the features in network. Furthermore, a novel block with FN is developed to improve the network representation, termed as FNB. Residual-in-residual structure is proposed to form a very deep network, which groups FNBs with a long skip connection for better information delivery and stabling the training phase. Extensive experimental results on testing benchmarks with bicubic (BI) degradation show our ISRN can not only recover more structural information, but also achieve competitive or better PSNR/SSIM results with much fewer parameters compared to other works. Besides BI, we simulate the real-world degradation with blur-downscale (BD) and downscale-noise (DN). ISRN and its extension ISRN+ both achieve better performance than others with BD and DN degradation models.

Figures (15)

• Figure 1: Visual quality comparisons for various image SR methods.
• Figure 2: A simple illustration of our proposed iterative scheme. The distance shrinks on LR space for each iteration, and the results are optimized on HR space.
• Figure 3: The network structure of the proposed iterative super-resolution network (ISRN). There are four components in ISRN: Solver SR, Solver LR, Down-Sampler, and Solver MLE, corresponding to different steps in formulation study. Solver SR is shared for each iteration to find the suitable mapping from LR space to HR space.
• Figure 4: Framework of the Solver SR and its components. There are four modules in Solver SR, mapping the features from LR space onto HR space.
• Figure 5: Illustration of the Down-sampler with different scaling factors.