Learning Image Demoireing from Unpaired Real Data

TL;DR

This work tackles image demoiréing by learning from unpaired real data rather than relying on paired moiré–clean image sets. It introduces UnDeM, a three-stage pipeline that first preprocesses images to group moiré patches by complexity, then uses a moiré-synthesis network with adversarial training to generate pseudo moiré patches paired with clean patches, and finally applies an adaptive denoise step to filter low-quality samples. The synthesized pseudo moiré enables training of off-the-shelf demoiréing models, yielding substantial improvements on FHDMi and UHDM compared with shooting-simulation and cyclic-learning baselines. The results demonstrate the practical value of unpaired real-data-driven synthesis for real-world demoiréing and provide a new data-generation paradigm for the demoiréing community, with potential applicability to broader pattern-agnostic image restoration tasks. $K=4$ groups, adversarial training with $E^m$, $G^m$, $D^m$, $E^c$, and the percentile-based adaptive denoise are key components that together address the domain gap between synthetic and real moiré patterns.

Abstract

This paper focuses on addressing the issue of image demoireing. Unlike the large volume of existing studies that rely on learning from paired real data, we attempt to learn a demoireing model from unpaired real data, i.e., moire images associated with irrelevant clean images. The proposed method, referred to as Unpaired Demoireing (UnDeM), synthesizes pseudo moire images from unpaired datasets, generating pairs with clean images for training demoireing models. To achieve this, we divide real moire images into patches and group them in compliance with their moire complexity. We introduce a novel moire generation framework to synthesize moire images with diverse moire features, resembling real moire patches, and details akin to real moire-free images. Additionally, we introduce an adaptive denoise method to eliminate the low-quality pseudo moire images that adversely impact the learning of demoireing models. We conduct extensive experiments on the commonly-used FHDMi and UHDM datasets. Results manifest that our UnDeM performs better than existing methods when using existing demoireing models such as MBCNN and ESDNet-L. Code: https://github.com/zysxmu/UnDeM

Figures (11)

• Figure 1: Illustration of image moiré. Natural moiré patterns are complex in varying thicknesses, frequencies, layouts, and colors across images and within an image.
• Figure 2: Visual examples of (a) real moiré-free images; (b) pseudo moiré images by shooting simulation niu2021morie; (c) pseudo moiré images by cyclic learning park2022unpaired; (d) pseudo moiré images by our UnDeM. Compared with details-missing and inauthentic pseudo moiré images by shooting simulation and park2022unpaired, ours result in more diverse moiré patterns and preserve more details of the moiré-free images. Best view by zooming in.
• Figure 3: Image preprocessing. Both moiré images $I^m$ and unpaired moiré-free images $I^f$ are split into patches. Patches from moiré images are further grouped in compliance with the complexity of moiré patterns.
• Figure 4: An illustration of moiré images of each group. Each group has its own moiré patterns complexity.
• Figure 5: Framework of our moiré synthesis network.