Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FC3DNet: A Fully Connected Encoder-Decoder for Efficient Demoir'eing

Zhibo Du, Long Peng, Yang Wang, Yang Cao, Zheng-Jun Zha

TL;DR

A Multi-Feature Multi-Attention Fusion (MFMAF) module to weigh the importance of each feature and compress them for efficiency enables the network to achieve performance comparable to state-of-the-art (SOTA) methods in real-world datasets while utilizing only a fraction of parameters, FLOPs, and runtime.

Abstract

Moiré patterns are commonly seen when taking photos of screens. Camera devices usually have limited hardware performance but take high-resolution photos. However, users are sensitive to the photo processing time, which presents a hardly considered challenge of efficiency for demoiréing methods. To balance the network speed and quality of results, we propose a \textbf{F}ully \textbf{C}onnected en\textbf{C}oder-de\textbf{C}oder based \textbf{D}emoiréing \textbf{Net}work (FC3DNet). FC3DNet utilizes features with multiple scales in each stage of the decoder for comprehensive information, which contains long-range patterns as well as various local moiré styles that both are crucial aspects in demoiréing. Besides, to make full use of multiple features, we design a Multi-Feature Multi-Attention Fusion (MFMAF) module to weigh the importance of each feature and compress them for efficiency. These designs enable our network to achieve performance comparable to state-of-the-art (SOTA) methods in real-world datasets while utilizing only a fraction of parameters, FLOPs, and runtime.

FC3DNet: A Fully Connected Encoder-Decoder for Efficient Demoir'eing

TL;DR

A Multi-Feature Multi-Attention Fusion (MFMAF) module to weigh the importance of each feature and compress them for efficiency enables the network to achieve performance comparable to state-of-the-art (SOTA) methods in real-world datasets while utilizing only a fraction of parameters, FLOPs, and runtime.

Abstract

Moiré patterns are commonly seen when taking photos of screens. Camera devices usually have limited hardware performance but take high-resolution photos. However, users are sensitive to the photo processing time, which presents a hardly considered challenge of efficiency for demoiréing methods. To balance the network speed and quality of results, we propose a \textbf{F}ully \textbf{C}onnected en\textbf{C}oder-de\textbf{C}oder based \textbf{D}emoiréing \textbf{Net}work (FC3DNet). FC3DNet utilizes features with multiple scales in each stage of the decoder for comprehensive information, which contains long-range patterns as well as various local moiré styles that both are crucial aspects in demoiréing. Besides, to make full use of multiple features, we design a Multi-Feature Multi-Attention Fusion (MFMAF) module to weigh the importance of each feature and compress them for efficiency. These designs enable our network to achieve performance comparable to state-of-the-art (SOTA) methods in real-world datasets while utilizing only a fraction of parameters, FLOPs, and runtime.
Paper Structure (14 sections, 5 equations, 5 figures, 2 tables)

This paper contains 14 sections, 5 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. PROPOSED METHOD
  3. Overall Network Architecture
  4. Encoder Block
  5. Decoder Block
  6. Multi-Feature Multi-Attention Fusion Module (MFMAF)
  7. Loss Functions
  8. EXPERIMENTAL RESULTS
  9. Datasets and Implementation Details
  10. Quantitative Analysis
  11. Qualitative Analysis
  12. Efficiency Study
  13. Ablation Study
  14. CONCLUSIONS

Figures (5)

  • Figure 1: Example of a moiré image and its ground-truth. The colors and shapes of moiré patterns vary in different areas of the image. $X_{1p}$, $X_{2p}$ and $X_{3p}$ are patches of $X_1$, $X_2$ and $X_3$, respectively, which are defined in Fig. \ref{['fig:nets']} and extracted from (a). They represent different characteristics.
  • Figure 2: Architecture of the proposed FC3DNet and structure of Encoder Block and Decoder Block.
  • Figure 3: Structure of MFMAF.
  • Figure 4: Qualitative comparison with classic methods on the UHDM dataset. Red boxes show zoom-in regions for demonstrating better details.
  • Figure 5: Comparison on efficiency. PSNR and runtime are tested on the FHDMi dataset using a RTX 3080 GPU. Areas of circles denote the parameter number of networks. MopNet and FHD$^2$eNet are exclusive due to ordinary performance and overlong runtime.