Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniDemoiré: Towards Universal Image Demoiréing with Data Generation and Synthesis

Zemin Yang, Yujing Sun, Xidong Peng, Siu Ming Yiu, Yuexin Ma

TL;DR

UniDemoiré tackles the challenging problem of generalizing image demoiréing across diverse moiré patterns by adopting a data-centric strategy. It combines a large-scale real moiré Pattern Dataset captured on a white background, a diffusion-based Moiré Pattern Generator, and a Moiré Image Synthesis pipeline with a Tone Refinement Network to produce realistic, diverse training samples. The approach delivers strong zero-shot and cross-domain demoiréing performance, outperforming state-of-the-art synthesis methods and enabling broader applicability. By enabling generation of vast realistic moiré data, UniDemoiré has high potential to scale demoiréing capabilities for real-world deployment and archival tasks.

Abstract

Image demoiréing poses one of the most formidable challenges in image restoration, primarily due to the unpredictable and anisotropic nature of moiré patterns. Limited by the quantity and diversity of training data, current methods tend to overfit to a single moiré domain, resulting in performance degradation for new domains and restricting their robustness in real-world applications. In this paper, we propose a universal image demoiréing solution, UniDemoiré, which has superior generalization capability. Notably, we propose innovative and effective data generation and synthesis methods that can automatically provide vast high-quality moiré images to train a universal demoiréing model. Our extensive experiments demonstrate the cutting-edge performance and broad potential of our approach for generalized image demoiréing.

UniDemoiré: Towards Universal Image Demoiréing with Data Generation and Synthesis

TL;DR

UniDemoiré tackles the challenging problem of generalizing image demoiréing across diverse moiré patterns by adopting a data-centric strategy. It combines a large-scale real moiré Pattern Dataset captured on a white background, a diffusion-based Moiré Pattern Generator, and a Moiré Image Synthesis pipeline with a Tone Refinement Network to produce realistic, diverse training samples. The approach delivers strong zero-shot and cross-domain demoiréing performance, outperforming state-of-the-art synthesis methods and enabling broader applicability. By enabling generation of vast realistic moiré data, UniDemoiré has high potential to scale demoiréing capabilities for real-world deployment and archival tasks.

Abstract

Image demoiréing poses one of the most formidable challenges in image restoration, primarily due to the unpredictable and anisotropic nature of moiré patterns. Limited by the quantity and diversity of training data, current methods tend to overfit to a single moiré domain, resulting in performance degradation for new domains and restricting their robustness in real-world applications. In this paper, we propose a universal image demoiréing solution, UniDemoiré, which has superior generalization capability. Notably, we propose innovative and effective data generation and synthesis methods that can automatically provide vast high-quality moiré images to train a universal demoiréing model. Our extensive experiments demonstrate the cutting-edge performance and broad potential of our approach for generalized image demoiréing.

Paper Structure

This paper contains 53 sections, 18 equations, 17 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Restoration and Demoiréing
  4. Moiré Image Synthesis
  5. Moiré Dataset
  6. Method
  7. Overview
  8. Moiré Pattern Dataset
  9. Capturing Process
  10. Data Diversity
  11. Moiré Pattern Generation
  12. Multi-Scale Cropping
  13. Colorfulness-Sharpness Selection
  14. Learning Moiré Patterns in the Latent Space
  15. Moiré Image Synthesis
  16. ...and 38 more sections

Figures (17)

  • Figure 1: The workflow of our proposed UniDemoiré.
  • Figure 2: Data collection setup (left), and examples of moiré patterns in our dataset captured at different zoom rates and screen panel (middle), and our generated patterns (right).
  • Figure 3: Data preprocessing for moiré pattern generation.
  • Figure 4: Overview of the Moiré Image Synthesis stage (a). It involves a Moiré Image Blending module (b) for initial moiré image synthesis and a Tone Refinement Network (c) to refine for more realistic results.
  • Figure 5: Visualization of our intermediate synthetic results.
  • ...and 12 more figures