Compressing Deep Image Super-resolution Models

TL;DR

This work employs a three-stage workflow for compressing deep SR models which significantly reduces their memory requirement, and achieves an 89% and 96% reduction in both model size and floating-point operations (FLOPs) respectively, compared to their original versions.

Abstract

Deep learning techniques have been applied in the context of image super-resolution (SR), achieving remarkable advances in terms of reconstruction performance. Existing techniques typically employ highly complex model structures which result in large model sizes and slow inference speeds. This often leads to high energy consumption and restricts their adoption for practical applications. To address this issue, this work employs a three-stage workflow for compressing deep SR models which significantly reduces their memory requirement. Restoration performance has been maintained through teacher-student knowledge distillation using a newly designed distillation loss. We have applied this approach to two popular image super-resolution networks, SwinIR and EDSR, to demonstrate its effectiveness. The resulting compact models, SwinIRmini and EDSRmini, attain an 89% and 96% reduction in both model size and floating-point operations (FLOPs) respectively, compared to their original versions. They also retain competitive super-resolution performance compared to their original models and other commonly used SR approaches. The source code and pre-trained models for these two lightweight SR approaches are released at https://pikapi22.github.io/CDISM/.

Figures (16)

• Figure 1: The proposed workflow for compressing SR networks. The pruning process obtains the corresponding compact model $\hat{M}$ from its intricate counterpart $M$. Here $N_c$, $N_l$ and $N_b$ represent the channel number, the layer number and the block number, respectively. $\hat{N}_c$, $\hat{N}_l$ and $\hat{N}_b$ denote the same but for the model after compression. $\mathcal{L}_{prune}\{.\}$ and $\mathcal{L}_{dis}\{.\}$ are the loss functions used in pruning and knowledge distillation processes.
• Figure 2: The basic blueprint of a modern image SR network.
• Figure 4: (Top) Average PSNR scores on Set5 and Set14 for the models presented in Table \ref{['tbl1']} versus their corresponding numbers of model parameters. Results for our two compact models are marked with solid symbols. (Bottom) The plot between the performance and FLOPs based on Set5 and Set14.
• Figure : butterfly (x2)
• Figure : butterfly (x2)