Single-image reflection removal via self-supervised diffusion models

TL;DR

This work tackles single-image reflection removal without requiring paired training data by marrying cycle-consistency with denoising diffusion probabilistic models (DDPM). It introduces a Reflective Removal Network (RRN) to decompose a captured image into transmission and reflection, and a Reflective Synthesis Network (RSN) to re-create the input with a nonlinear attention-based fusion, aided by a Transmission Discriminator for realism. A new Museum Reflection Removal (MRR) dataset and the Reflection Artifact Measure (RAM) are proposed to broaden evaluation and quantify residual artifacts. Across real and synthetic benchmarks (SIR^2, FRR, MRR), the proposed method achieves state-of-the-art performance with significant gains in PSNR, SSIM, LPIPS, and RAM, demonstrating strong potential for applications in heritage preservation and digital archiving.

Abstract

Reflections often degrade the visual quality of images captured through transparent surfaces, and reflection removal methods suffers from the shortage of paired real-world samples.This paper proposes a hybrid approach that combines cycle-consistency with denoising diffusion probabilistic models (DDPM) to effectively remove reflections from single images without requiring paired training data. The method introduces a Reflective Removal Network (RRN) that leverages DDPMs to model the decomposition process and recover the transmission image, and a Reflective Synthesis Network (RSN) that re-synthesizes the input image using the separated components through a nonlinear attention-based mechanism. Experimental results demonstrate the effectiveness of the proposed method on the SIR$^2$, Flash-Based Reflection Removal (FRR) Dataset, and a newly introduced Museum Reflection Removal (MRR) dataset, showing superior performance compared to state-of-the-art methods.

Figures (11)

• Figure 1: Real-world reflections can be highly complex, often involving multiple superimposed reflections and blurring phenomena. Most methods trained on simple synthetic dataset are struggle to reconstruct transmission images in the wild.
• Figure 2: The self-supervised diffusion model has three main components: Reflective Removal Network, Reflective Synthesis Network, and Transmission Discriminator.
• Figure 3: Dual DDPM ensures accurate transmission and reflection image predictions.
• Figure 4: Reflective synthesis network restores the original image from transmission and reflection image with attention mechanism.
• Figure 5: The Museum Reflection Removal Dataset includes exhibition samples with reflections from various fields.