Photometric Correction for Infrared Sensors

TL;DR

This work addresses the challenge of reconstructing 3D structure from infrared imagery by accounting for the distinct photometric response of microbolometer sensors. It introduces a heating/cooling based photometric correction model, integrates it into a Direct Sparse Odometry–style SfM pipeline, and optimizes the heating and cooling time constants $\tau_h$ and $\tau_c$ within a complete video sensing framework. Empirical results on FLIR ADAS and BU-TIV datasets demonstrate that IR photometric correction (IR+cor) yields denser feature tracks, more accurate road-plane reconstructions, and more stable camera trajectories than IR or RGB approaches. The findings indicate that infrared photometric correction can achieve reconstruction quality competitive with RGB-based SfM and potentially generalize to other IR vision tasks under challenging illumination or thermal conditions.

Abstract

Infrared thermography has been widely used in several domains to capture and measure temperature distributions across surfaces and objects. This methodology can be further expanded to 3D applications if the spatial distribution of the temperature distribution is available. Structure from Motion (SfM) is a photometric range imaging technique that makes it possible to obtain 3D renderings from a cloud of 2D images. To explore the possibility of 3D reconstruction via SfM from infrared images, this article proposes a photometric correction model for infrared sensors based on temperature constancy. Photometric correction is accomplished by estimating the scene irradiance as the values from the solution to a differential equation for microbolometer pixel excitation with unknown coefficients and initial conditions. The model was integrated into an SfM framework and experimental evaluations demonstrate the contribution of the photometric correction for improving the estimates of both the camera motion and the scene structure. Further, experiments show that the reconstruction quality from the corrected infrared imagery achieves performance on par with state-of-the-art reconstruction using RGB sensors.

Figures (6)

• Figure 1: An illustration of microbolometer pixels' behavior during the heating and cooling process. (a) Pixels do not have enough time to reach the temperature of the scene being measured. (b) In two (or more) consecutive frames the later frame(s) has a memory of the energy residual in the previous frame(s).
• Figure 2: An image pair sample from the FLIR ADAS dataset: IR image (left) and RGB image (right).
• Figure 3: The number of tracked features is plotted for each video frame for "RGB" (blue), "IR" (red), and "IR+cor" (yellow) SfM estimates. The plot shows that the proposed IR photometric correction allows more stable feature detection which results in more tracked features leading to denser SfM estimates.
• Figure 4: Qualitative examples. (a) Infrared image from the FLIR ADAS Dataset. (b) "IR" point cloud reconstruction. (c) "IR+cor" point cloud reconstruction. (d) RGB image of the same scene from the dataset. (e) "RGB" point cloud reconstruction.
• Figure 5: Histogram and approximated Gaussian distributions of the fitting error of SfM-estimated 3D points to a common planar surface (the road plane). RGB (green) and IR (blue) show more error variance than the proposed IR photometric correction approach (purple).