Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MFDNet: Multi-Frequency Deflare Network for Efficient Nighttime Flare Removal

Yiguo Jiang, Xuhang Chen, Chi-Man Pun, Shuqiang Wang, Wei Feng

TL;DR

A lightweight Multi-Frequency Deflare Network (MFDNet) based on the Laplacian Pyramid that outperforms state-of-the-art methods in removing nighttime flare on real-world and synthetic images from the Flare7K dataset and can reduce the computational cost by processing in multiple frequency bands instead of directly removing the flare on the input image.

Abstract

When light is scattered or reflected accidentally in the lens, flare artifacts may appear in the captured photos, affecting the photos' visual quality. The main challenge in flare removal is to eliminate various flare artifacts while preserving the original content of the image. To address this challenge, we propose a lightweight Multi-Frequency Deflare Network (MFDNet) based on the Laplacian Pyramid. Our network decomposes the flare-corrupted image into low and high-frequency bands, effectively separating the illumination and content information in the image. The low-frequency part typically contains illumination information, while the high-frequency part contains detailed content information. So our MFDNet consists of two main modules: the Low-Frequency Flare Perception Module (LFFPM) to remove flare in the low-frequency part and the Hierarchical Fusion Reconstruction Module (HFRM) to reconstruct the flare-free image. Specifically, to perceive flare from a global perspective while retaining detailed information for image restoration, LFFPM utilizes Transformer to extract global information while utilizing a convolutional neural network to capture detailed local features. Then HFRM gradually fuses the outputs of LFFPM with the high-frequency component of the image through feature aggregation. Moreover, our MFDNet can reduce the computational cost by processing in multiple frequency bands instead of directly removing the flare on the input image. Experimental results demonstrate that our approach outperforms state-of-the-art methods in removing nighttime flare on real-world and synthetic images from the Flare7K dataset. Furthermore, the computational complexity of our model is remarkably low.

MFDNet: Multi-Frequency Deflare Network for Efficient Nighttime Flare Removal

TL;DR

A lightweight Multi-Frequency Deflare Network (MFDNet) based on the Laplacian Pyramid that outperforms state-of-the-art methods in removing nighttime flare on real-world and synthetic images from the Flare7K dataset and can reduce the computational cost by processing in multiple frequency bands instead of directly removing the flare on the input image.

Abstract

When light is scattered or reflected accidentally in the lens, flare artifacts may appear in the captured photos, affecting the photos' visual quality. The main challenge in flare removal is to eliminate various flare artifacts while preserving the original content of the image. To address this challenge, we propose a lightweight Multi-Frequency Deflare Network (MFDNet) based on the Laplacian Pyramid. Our network decomposes the flare-corrupted image into low and high-frequency bands, effectively separating the illumination and content information in the image. The low-frequency part typically contains illumination information, while the high-frequency part contains detailed content information. So our MFDNet consists of two main modules: the Low-Frequency Flare Perception Module (LFFPM) to remove flare in the low-frequency part and the Hierarchical Fusion Reconstruction Module (HFRM) to reconstruct the flare-free image. Specifically, to perceive flare from a global perspective while retaining detailed information for image restoration, LFFPM utilizes Transformer to extract global information while utilizing a convolutional neural network to capture detailed local features. Then HFRM gradually fuses the outputs of LFFPM with the high-frequency component of the image through feature aggregation. Moreover, our MFDNet can reduce the computational cost by processing in multiple frequency bands instead of directly removing the flare on the input image. Experimental results demonstrate that our approach outperforms state-of-the-art methods in removing nighttime flare on real-world and synthetic images from the Flare7K dataset. Furthermore, the computational complexity of our model is remarkably low.
Paper Structure (23 sections, 6 equations, 11 figures, 3 tables)

This paper contains 23 sections, 6 equations, 11 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Restoration
  4. Flare Removal
  5. Traditional methods
  6. Deep learning-based methods
  7. Methodology
  8. Overview
  9. Low-Frequency Flare Perception Module
  10. Hierarchical Fusion Reconstruction Module
  11. Loss Function
  12. Experiments and Results
  13. Setup
  14. Dataset
  15. Implementation Details
  16. ...and 8 more sections

Figures (11)

  • Figure 1: Our MFDNet achieves the state-of-the-art performance on nighttime flare removal task while being computationally efficient.
  • Figure 2: The structure of our proposed MFDNet. Given a nighttime flare-corrupted image, our MFDNet first decouples its content and illumination information by decomposing this image into high and low-frequency bands. Then MFDNet performs flare removal in the low- frequency part of the image, followed by gradual fusion with the detailed high-frequency part to reconstruct the final flare-free image.
  • Figure 3: The structure of Feature Extraction Transformer Block (FETB) and Feature Refinement Transformer Block (FRTB).
  • Figure 4: The structure of Feature Encoder Convolution Block (FECB) and Feature Decoder Convolution Block (FDCB).
  • Figure 5: The structure of Feature Aggregation Block (FAB).
  • ...and 6 more figures