PolarFree: Polarization-based Reflection-free Imaging

TL;DR

This work tackles polarization-based reflection removal by introducing PolaRGB, a large-scale real-world RGB-polarization dataset with ground-truth transmission and polarization cues, and PolarFree, a two-stage diffusion-guided network that generates a polarization-informed prior to aid accurate reflection separation. By leveraging Stokes parameters, AoLP, and DoLP, the method exploits physics-based polarization cues to distinguish transmission from reflection, while a phase-based loss mitigates color distortions inherent to semi-reflections. The approach shows quantitative gains over priors and baselines on PolaRGB, and qualitative robustness in real-world scenes such as museums, demonstrating practical applicability. Overall, PolaRGB and PolarFree establish a new benchmark for polarization-based reflection removal and enable more reliable, real-world imaging through semi-reflective surfaces.

Abstract

Reflection removal is challenging due to complex light interactions, where reflections obscure important details and hinder scene understanding. Polarization naturally provides a powerful cue to distinguish between reflected and transmitted light, enabling more accurate reflection removal. However, existing methods often rely on small-scale or synthetic datasets, which fail to capture the diversity and complexity of real-world scenarios. To this end, we construct a large-scale dataset, PolaRGB, for Polarization-based reflection removal of RGB images, which enables us to train models that generalize effectively across a wide range of real-world scenarios. The PolaRGB dataset contains 6,500 well-aligned mixed-transmission image pairs, 8x larger than existing polarization datasets, and is the first to include both RGB and polarization images captured across diverse indoor and outdoor environments with varying lighting conditions. Besides, to fully exploit the potential of polarization cues for reflection removal, we introduce PolarFree, which leverages diffusion process to generate reflection-free cues for accurate reflection removal. Extensive experiments show that PolarFree significantly enhances image clarity in challenging reflective scenarios, setting a new benchmark for polarized imaging and reflection removal. Code and dataset are available at https://github.com/mdyao/PolarFree.

Figures (9)

• Figure 1: Our PolarFree effectively leverages polarization information to remove reflections, achieving superior performance in challenging scenes with complex backgrounds and highlights where previous methods kim2020singlehu2023singlezhu2024revisiting often fail.
• Figure 2: (a) & (b) A semi-reflector transforms unpolarized light into polarized light upon reflection and refraction, which is undetectable by standard RGB cameras but can be leveraged by polarization cameras for reflection-suppression tasks. (c) At the Brewster angle born2013principles, a polarizer minimizes reflections.
• Figure 3: Overview of the PolaRGB dataset. (a) Hierarchical structure of scenes is shown in the ring, with legends indicating sample counts and subset types. (b) Typical scenes illustrating varied reflection conditions: I. smooth blending of reflection and refraction, II. abrupt reflection with mixed components, III. reflection dominant over transmission, and IV. minimal or no reflection. (c) Video-based capture method (details in Sec. \ref{['sec:PolaRGB Dataset']}). (d) We provide polarized images at angles $\phi$ = 0$^\circ$, 45$^\circ$, 90$^\circ$, and 135$^\circ$, along with derived AoLP, DoLP, and a well-aligned unpolarized image. The dataset also includes ground truth transmission and estimated reflections, all available in both raw and RGB formats.
• Figure 4: Data processing pipeline for obtaining aligned mixed and transmission images, and polarized images.
• Figure 5: Pipeline of PolarFree. (a) During inference, PolarFree leverages polarized and RGB images as inputs, which are feeds into a conditional diffusion model to generate the prior $\hat{z}_0$. The generated prior, along with the inputs, is then passed to the reflection removal backbone $\mathcal{F}_{\text{remove}}$ to remove reflections. (b) PolarFree is trained in two stages. (1) A prior encoder extracts a reflection-free prior $z_0$ from clean transmission images and polarization cues, which serves as the supervision for the conditional diffusion model in stage two. (2) The conditional diffusion model is trained to progressively denoise noisy images, supervised by the prior from stage one, ensuring robust reflection separation.