Pola4All: survey of polarimetric applications and an open-source toolkit to analyze polarization

TL;DR

Polarization imaging provides rich cues for material, geometry, and scene understanding, particularly in challenging scenarios where RGB cues fail. The paper surveys recent polarization-based applications in vision and robotics and introduces Pola4All, an open-source, ROS-compatible toolkit with a GUI that unifies acquisition, calibration, and processing for RGB-polarization cameras. It details polarization theory (Stokes/DoLP/AoLP), DoFP sensing, and both model- and data-driven approaches across enhancement, segmentation, depth/normal estimation, and pose tasks, while highlighting the lack of standard datasets. The toolkit offers end-to-end functionality, including raw data handling, white balance, calibration, and polarization-specific processing algorithms, aiming to lower barriers to adoption and foster community-driven growth in polarimetric vision.

Abstract

Polarization information of the light can provide rich cues for computer vision and scene understanding tasks, such as the type of material, pose, and shape of the objects. With the advent of new and cheap polarimetric sensors, this imaging modality is becoming accessible to a wider public for solving problems such as pose estimation, 3D reconstruction, underwater navigation, and depth estimation. However, we observe several limitations regarding the usage of this sensorial modality, as well as a lack of standards and publicly available tools to analyze polarization images. Furthermore, although polarization camera manufacturers usually provide acquisition tools to interface with their cameras, they rarely include processing algorithms that make use of the polarization information. In this paper, we review recent advances in applications that involve polarization imaging, including a comprehensive survey of recent advances on polarization for vision and robotics perception tasks. We also introduce a complete software toolkit that provides common standards to communicate with and process information from most of the existing micro-grid polarization cameras on the market. The toolkit also implements several image processing algorithms for this modality, and it is publicly available on GitHub: https://github.com/vibot-lab/Pola4all_JEI_2023.

Figures (13)

• Figure 1: (a) Micro-grid sensor scheme with four super-pixels with polarizer orientations of $0^\circ$, $45^\circ$, $90^\circ$, and $135^\circ$ arranged following a Bayer pattern of a typical RGB-polarization sensor our_calib_paper. (b) Sketch of the interaction between a light ray coming from the air with refractive index $n_{1}$ and an object with refractive index $n_{2}$. When the incident light ray hits a point on the surface of the object, a portion of the light is reflected and a portion of it is refracted or transmitted in the second medium. The law of reflection states that the angle of reflection is equal to the angle of incidence $\alpha_{1}$. Regarding the refracted light, its direction of travel will change, and it will be equal to $\alpha_{2}$.
• Figure 2: Results of some of the reviewed algorithms for image enhancement. From left to right: (a) Input image, ground truth, and algorithm results of DoLP-based color constancy CVPR_2022_dolp_color_const. (b) Input image, obtained diffuse image, and obtained specular image of Polarization-guided specular reflection separation PolaDiffuseSpecSep. (c) Input image, ground truth, and resulting images of the polarization dehazing method dehazingpaper. Images courtesy of the respective works.
• Figure 3: Segmentation examples with polarization cues. From left to right: (a) Input image, results obtained with an RGB-only method, and the result of glass Segmentation using intensity and spectral polarization informationglass_segm_paper. (b) Input image with transparent objects, RGB-only method object segmentation results, and results obtained with the polarization data-driven methodTransparentObjectSegmentation. (c) Input image, RGB-only method results, and the results of the Multimodal Material Segmentation algorithm CVPR2022MutiModMatSeg. Images courtesy of the respective works.
• Figure 4: Results of some of the reviewed algorithms for surface normal and depth estimation. From left to right: (a) Intensity input image, reconstructed mesh, and estimated normal map by the method introduced by PolaDenseMapping. (b) Real RGB image, corresponding rendered image, and the corresponding rendered degree of linear polarization. This result corresponds to the scene rendering technique implemented in pBRDF. (c) Input polarization images, ground truth, and estimated normal maps. These results are courtesy of ECCV_DeepSfP_Ba.
• Figure 5: Results of some of the reviewed algorithms for pose estimation. From left to right: (a) Input RGB image, reconstructed surface with the algorithm provided in PolaRelPosPred. (b) Glass vase model, and the detected pose from two viewpoints. Results courtesy of Gao_Pola_pose_pred.