Learning to Remove Wrinkled Transparent Film with Polarized Prior

TL;DR

This paper first physically model the imaging of industrial materials covered by the film, then builds a practical dataset with polarization information containing paired data with and without transparent film to remove interference from the film with an end-to-end framework.

Abstract

In this paper, we study a new problem, Film Removal (FR), which attempts to remove the interference of wrinkled transparent films and reconstruct the original information under films for industrial recognition systems. We first physically model the imaging of industrial materials covered by the film. Considering the specular highlight from the film can be effectively recorded by the polarized camera, we build a practical dataset with polarization information containing paired data with and without transparent film. We aim to remove interference from the film (specular highlights and other degradations) with an end-to-end framework. To locate the specular highlight, we use an angle estimation network to optimize the polarization angle with the minimized specular highlight. The image with minimized specular highlight is set as a prior for supporting the reconstruction network. Based on the prior and the polarized images, the reconstruction network can decouple all degradations from the film. Extensive experiments show that our framework achieves SOTA performance in both image reconstruction and industrial downstream tasks. Our code will be released at \url{https://github.com/jqtangust/FilmRemoval}.

Figures (14)

• Figure 1: The Red box presents a challenge in industrial recognition systems, where the product information is often hidden beneath the wrinkled transparent film. The Green box is the image we expect to generate, with the film layer removed. Removing the wrinkled film makes the information on industrial material clearer.
• Figure 2: Wrinkled Transparent Film Model. (A) The polarized image. (B) The 3D physics model of the local region. The light is reflected through the transparent wrinkled film and captured by the polarization camera. (C) The light path diagram. Polarized cameras capture two components: Specular Reflection ($I_h$), and Diffuse Reflection ($I_{md}$). The Original Diffuse Reflection ($I_{m}$) would be interfered by various degradations ($I_{md} - I_{m}$) from the film.
• Figure 3: Model of elliptically polarized light. $E$ represents polarized light at any angle, which can be calculated by this model. $I_{max}$ and $I_{min}$ are two components indicating the maximum and minimum intensity of elliptically polarized light.
• Figure 4: Prototype of industrial optical photography pipeline. We have built an optical pipeline for capturing the dataset in the industrial environment. As objects traverse the objective pipeline, the polarizing camera captures images continuously. Subsequently, the acquired data is sent to the monitor for pre-processing.
• Figure 5: Two Decoupling Components. Specular highlight, $I_{h}$ and other degradations, $I_{d}$. The Red box shows the degradations, the Green box is the Ground Truth.