Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-scale frequency separation network for image deblurring

Yanni Zhang, Qiang Li, Miao Qi, Di Liu, Jun Kong, Jianzhong Wang

TL;DR

A new method called multi-scale frequency separation network (MSFS-Net) for image deblurring, which introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales.

Abstract

Image deblurring aims to restore the detailed texture information or structures from blurry images, which has become an indispensable step in many computer vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales. Then, a cycle-consistency strategy and a contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a cross-scale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.

Multi-scale frequency separation network for image deblurring

TL;DR

A new method called multi-scale frequency separation network (MSFS-Net) for image deblurring, which introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales.

Abstract

Image deblurring aims to restore the detailed texture information or structures from blurry images, which has become an indispensable step in many computer vision tasks. Although various methods have been proposed to deal with the image deblurring problem, most of them treated the blurry image as a whole and neglected the characteristics of different image frequencies. In this paper, we present a new method called multi-scale frequency separation network (MSFS-Net) for image deblurring. MSFS-Net introduces the frequency separation module (FSM) into an encoder-decoder network architecture to capture the low- and high-frequency information of image at multiple scales. Then, a cycle-consistency strategy and a contrastive learning module (CLM) are respectively designed to retain the low-frequency information and recover the high-frequency information during deblurring. At last, the features of different scales are fused by a cross-scale feature fusion module (CSFFM). Extensive experiments on benchmark datasets show that the proposed network achieves state-of-the-art performance.
Paper Structure (23 sections, 7 equations, 7 figures, 3 tables)

This paper contains 23 sections, 7 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Image Deblurring
  4. Frequency Separation
  5. Contrastive Learning
  6. Method
  7. Motivation
  8. Overview
  9. Frequency Separation Module
  10. Cross-scale Feature Fusion Module
  11. Contrastive Learning Module
  12. Loss Functions
  13. Multi-scale Consistent Loss for Low-frequency Features
  14. Multi-scale Contrastive Loss for High-frequency Features
  15. The Final Loss of MSFS-Net
  16. ...and 8 more sections

Figures (7)

  • Figure 1: The distributions of entropy obtained by samples in GoPro dataset. In this figure, the LF component of each image is obtained by a low-pass Gaussian filter and the HF component is obtained by subtracting the LF component from the original image. Top: From left to right are the distributions of entropy obtained by HF components of sharp ($HF_{Sharp}$) and blurry ($HF_{Blur}$) images at original, 1/2 and 1/4 scales. Down: From left to right are the distributions of entropy obtained by LF components of sharp ($LF_{Sharp}$) and blurry ($LF_{Blur}$) images at original, 1/2 and 1/4 scales.
  • Figure 2: The architecture of the proposed MSFS-Net.
  • Figure 3: (a) The architecture of OctConv. (b) The architecture of frequency separation module (FSM).
  • Figure 4: The architecture of the cross-scale feature fusion module (CSFFM).
  • Figure 5: Visual comparison of the deblurring results on GoPro dataset.
  • ...and 2 more figures