Thermal Polarimetric Multi-view Stereo

TL;DR

This work tackles robust 3D shape reconstruction under challenging materials and lighting by leveraging long-wave infrared (LWIR) polarization, notably the angle of linear polarization ($AoLP$). It develops a unified LWIR polarization theory and an implicit neural surface approach that uses multi-view AoLP cues with differentiable rendering, incorporating tangent-space, silhouette, and Eikonal constraints to recover fine geometry as a Signed Distance Function. The key contributions are (i) showing LWIR polarization avoids the specular–diffuse ambiguities inherent to visible polarization, (ii) exploiting $AoLP$ as a material-robust cue for surface normals, and (iii) delivering a multi-view, neural-implicit reconstruction framework that achieves state-of-the-art detail on transparent and low-reflective objects. This approach promises illumination- and material-insensitive 3D reconstruction in challenging scenes, with practical impact for industrial inspection and robotics where visible cues are unreliable.

Abstract

This paper introduces a novel method for detailed 3D shape reconstruction utilizing thermal polarization cues. Unlike state-of-the-art methods, the proposed approach is independent of illumination and material properties. In this paper, we formulate a general theory of polarization observation and show that long-wave infrared (LWIR) polarimetric imaging is free from the ambiguities that affect visible polarization analyses. Subsequently, we propose a method for recovering detailed 3D shapes using multi-view thermal polarimetric images. Experimental results demonstrate that our approach effectively reconstructs fine details in transparent, translucent, and heterogeneous objects, outperforming existing techniques.

Figures (5)

• Figure 1: Coordinate systems and the zenith and projected azimuth angles of the surface normal. While Mueller calculus is performed in the object's surface coordinate system, the observed Stokes parameter and projected azimuth angle are represented in the image coordinate system.
• Figure 2: Visible and thermal polarization observations of spheres made from various materials under natural room lighting. While visible polarization is unstable due to variations in a material's optical properties, thermal polarimetric cues, especially AoLP images, remain consistent across different materials.
• Figure 3: Camera system and setup. The system consists of a rotation stage to rotate the target and a thermal camera with a wire-grid polarizer. A visible polarization camera is used for camera pose estimation and for comparison with the visible MVAS.
• Figure 4: Experimental results. AoLP images in both visible and LWIR spectra are shown below the scene photograph. Each method’s result includes the recovered shape, surface normals, and angular error maps. The Chamfer distance is displayed below the estimated shape and the mean angular error is displayed to the right of the angular error map. For both metrics, lower values indicate better performance. The smallest errors are shown in magenta. It demonstrates that the proposed method outperforms the other methods.
• Figure 5: Qualitative results for other objects. AoLP images are shown below the scene photos. For each method, the upper image is the estimated shape and the lower is the corresponding normal map. It is shown that our method successfully reconstructs fine details of the object, particularly the concave parts in the transparent vase and bottle, while other methods fail to reconstruct such details.