Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reflection Removal Using Recurrent Polarization-to-Polarization Network

Wenjiao Bian, Yusuke Monno, Masatoshi Okutomi

TL;DR

The paper tackles reflection removal under realistic polarization imaging by introducing RP2PN, a polarization-to-polarization, recurrent network. It processes polarized inputs $I_ heta$ to produce polarized reflection and transmission $R_ heta$ and $T_ heta$ across four orientations, and iteratively refines them via an R-LSTM-Net and a T-Net, aided by polarized difference images and DoP cues. Training leverages losses in polarized domains and a PNCC-based cross-correlation term, evaluated on Lei et al.'s real-world dataset, where RP2PN achieves state-of-the-art PSNR/SSIM and produces physically meaningful polarization outputs. The method demonstrates the practical benefit of preserving polarization information for more accurate and robust reflection separation in real scenes.

Abstract

This paper addresses reflection removal, which is the task of separating reflection components from a captured image and deriving the image with only transmission components. Considering that the existence of the reflection changes the polarization state of a scene, some existing methods have exploited polarized images for reflection removal. While these methods apply polarized images as the inputs, they predict the reflection and the transmission directly as non-polarized intensity images. In contrast, we propose a polarization-to-polarization approach that applies polarized images as the inputs and predicts "polarized" reflection and transmission images using two sequential networks to facilitate the separation task by utilizing the interrelated polarization information between the reflection and the transmission. We further adopt a recurrent framework, where the predicted reflection and transmission images are used to iteratively refine each other. Experimental results on a public dataset demonstrate that our method outperforms other state-of-the-art methods.

Reflection Removal Using Recurrent Polarization-to-Polarization Network

TL;DR

The paper tackles reflection removal under realistic polarization imaging by introducing RP2PN, a polarization-to-polarization, recurrent network. It processes polarized inputs to produce polarized reflection and transmission and across four orientations, and iteratively refines them via an R-LSTM-Net and a T-Net, aided by polarized difference images and DoP cues. Training leverages losses in polarized domains and a PNCC-based cross-correlation term, evaluated on Lei et al.'s real-world dataset, where RP2PN achieves state-of-the-art PSNR/SSIM and produces physically meaningful polarization outputs. The method demonstrates the practical benefit of preserving polarization information for more accurate and robust reflection separation in real scenes.

Abstract

This paper addresses reflection removal, which is the task of separating reflection components from a captured image and deriving the image with only transmission components. Considering that the existence of the reflection changes the polarization state of a scene, some existing methods have exploited polarized images for reflection removal. While these methods apply polarized images as the inputs, they predict the reflection and the transmission directly as non-polarized intensity images. In contrast, we propose a polarization-to-polarization approach that applies polarized images as the inputs and predicts "polarized" reflection and transmission images using two sequential networks to facilitate the separation task by utilizing the interrelated polarization information between the reflection and the transmission. We further adopt a recurrent framework, where the predicted reflection and transmission images are used to iteratively refine each other. Experimental results on a public dataset demonstrate that our method outperforms other state-of-the-art methods.
Paper Structure (9 sections, 6 equations, 5 figures, 2 tables)

This paper contains 9 sections, 6 equations, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Network Structure
  4. Loss Functions
  5. Experimental Results
  6. Implementation Details of Our RP2PN
  7. Comparison with Other Methods
  8. Ablation Study
  9. Conclusion

Figures (5)

  • Figure 1: Different input-output models for the reflection removal. (a) Both the input and the output of standard single-image methods are intensity images. (b) Existing polarization-based methods apply polarized images only to the input. (c) Our proposed polarization-to-polarization approach predicts the output reflection and transmission as polarized images as well.
  • Figure 2: The overall structure of our proposed RP2PN.
  • Figure 3: An example of $I_{diff}$ image. $I_{0,45}$ demonstrates closer features to the features of the ground-truth $R$ than either $I_0$ or $I_{45}$. This is due to the polarized $T$ component being considerably weaker than the polarized $R$ component. The brightness of $I_{0,45}$ is adjusted solely for the visualization purpose.
  • Figure 4: Qualitative comparison with existing methods.
  • Figure 5: Example of qualitative results on polarization outputs.