A Geometric Model for Polarization Imaging on Projective Cameras

Abstract

The vast majority of Shape-from-Polarization (SfP) methods work under the oversimplified assumption of using orthographic cameras. Indeed, it is still not well understood how to project the Stokes vectors when the incoming rays are not orthogonal to the image plane. We try to answer this question presenting a geometric model describing how a general projective camera captures the light polarization state. Based on the optical properties of a tilted polarizer, our model is implemented as a pre-processing operation acting on raw images, followed by a per-pixel rotation of the reconstructed normal field. In this way, all the existing SfP methods assuming orthographic cameras can behave like they were designed for projective ones. Moreover, our model is consistent with state-of-the-art forward and inverse renderers (like Mitsuba3 and ART), intrinsically enforces physical constraints among the captured channels, and handles demosaicing of DoFP sensors. Experiments on existing and new datasets demonstrate the accuracy of the model when applied to commercially available polarimetric cameras.

Figures (4)

• Figure 1: Sketch of the proposed model. In the camera reference system (left), the ray passing through the $j^{th}$ pixel (dashed line) is not propagating parallel to the $z$-axis. Therefore, the effect of a polarizer with angle $\alpha_i$ placed parallel to the image plane cannot be described with formulas used for orthographic cameras. To solve this, we define a local reference system for each ray (right) oriented along the direction of propagation. Since the linear polarizer is tilted, its effective angle $\hat{\alpha}_i^j$ is different from the actual orientation it has on the sensor. The tilted polarizer model korger2013polarization is used to compensate this distortion and recover the correct Stokes vector $S_j$ in the reference frame of the $j^{th}$ pixel. Finally, a rotation $R_j$ allow us to map normal vectors estimated from $S_j$ back to the camera reference system.
• Figure 2: Average per-pixel error in degrees of the produced AoLP for three models. Note that the color bar for the Orthographic model is substantially different to avoid saturation.
• Figure 3: Plane normal error distributions for our method and PPA.
• Figure 4: Qualitative comparison of the estimated normal field with methods Ortho (left), Our (center), and SfPW (right) on an instance of the PPA dataset. Errors are computed only inside the plane area but we kept the rest of the scene for visualization purposes. Note how a radial pattern is visible in the Ortho model since the distortion due to the ray orientation is not accounted for. SfPw performs poorly because it has been trained on a much noisier dataset and apparently is not able to generalize to simpler scenarios.