NTR-Gaussian: Nighttime Dynamic Thermal Reconstruction with 4D Gaussian Splatting Based on Thermodynamics

TL;DR

This work tackles the limitation of static 3D thermal reconstruction by introducing NTR-Gaussian, a nighttime dynamic framework that models temperature as infrared radiation through 4D Gaussian Splatting coupled with thermodynamics. It predicts emissivity $E$, convective heat transfer coefficient $C$, and heat capacity $H$ with neural nets and integrates differential thermodynamic equations to forecast scene temperatures across time. A dedicated nighttime NTR dataset supports evaluation of dynamic 3D thermal reconstruction from aerial TIR imagery paired with synthetic RGB views. Experimental results show the approach achieves temperature prediction errors within $1^\circ$C and outperforms static and other dynamic baselines, enabling more accurate, time-aware thermal analysis for applications like building monitoring and energy management. While promising, the method requires scene-specific optimization and currently omits humidity and wind effects, pointing to directions for broader generalization and environmental constraint integration.

Abstract

Thermal infrared imaging offers the advantage of all-weather capability, enabling non-intrusive measurement of an object's surface temperature. Consequently, thermal infrared images are employed to reconstruct 3D models that accurately reflect the temperature distribution of a scene, aiding in applications such as building monitoring and energy management. However, existing approaches predominantly focus on static 3D reconstruction for a single time period, overlooking the impact of environmental factors on thermal radiation and failing to predict or analyze temperature variations over time. To address these challenges, we propose the NTR-Gaussian method, which treats temperature as a form of thermal radiation, incorporating elements like convective heat transfer and radiative heat dissipation. Our approach utilizes neural networks to predict thermodynamic parameters such as emissivity, convective heat transfer coefficient, and heat capacity. By integrating these predictions, we can accurately forecast thermal temperatures at various times throughout a nighttime scene. Furthermore, we introduce a dynamic dataset specifically for nighttime thermal imagery. Extensive experiments and evaluations demonstrate that NTR-Gaussian significantly outperforms comparison methods in thermal reconstruction, achieving a predicted temperature error within 1 degree Celsius.

Figures (11)

• Figure 1: UVA platform and Cameras. DJI Matrice 300 RTK(Left) and DJI H20T(Right).
• Figure 2: Method Overview. Our method consists of two stages. In the first stage, we input the discrete time encoding from the true thermal temperature image corresponding to time $t$ and the position encoding obtained from the tilted photography model in the collected thermal region. We optimize Temp_Net, $\alpha$, $r$ and $s$ to obtain the Gaussian sphere infrared radiation temperature Temp($t$) and attribute values corresponding to discrete time. In the second stage, given the initial time $t_0$ and the true thermal temperature image corresponding to time $t$, divide this time interval into $N$ parts to obtain $N$ time encodings. Combine the position encoding through Temp_Net to obtain Temp($t_N$). Then, use Temp($t_N$), time encodings, position encoding, and semantic feature information obtained from Feature_GS zhou2024feature through Thermal_Net to obtain the emissivity E($t_N$), convective heat transfer coefficient C($t_N$), and heat capacity H($t_N$) of the Gaussian sphere for each moment in the corresponding time interval. By applying thermodynamic equations, we calculate the temperature variation $\Delta$T($t_N$) of the Gaussian sphere at each moment and integrate to obtain the temperature Temp_integral($t$) corresponding to time $t$. We freeze the Gaussian sphere parameters obtained in the first stage and render the thermal prediction images separately from Temp($t$) and Temp_integral($t$). By jointly optimizing the network, we ensure that these two values are consistent at each moment, allowing us to predict the infrared radiation temperature values at any moment in any nighttime scene based on physical laws.
• Figure 2: Area and route planning.
• Figure 3: Thermodynamic Theory. The temperature of outdoor objects shows a heat absorption and release performance during the day and night. During the day, the objects absorb heat from solar radiation, while at night, with the setting of the sun, the objects mainly release heat through their own radiation and convective heat exchange.
• Figure 3: Taking emissivity as an example, we looked up the table to obtain the relative relationships among the buildings, roads and trees in the figure. They are roads > trees > buildings respectively. Therefore, our normalized emissivity map is relatively accurate and has obvious advantages over the artificial settings in traditional methods.