TeX-NeRF: Neural Radiance Fields from Pseudo-TeX Vision

TL;DR

The paper tackles infrared-only 3D scene reconstruction and temperature estimation, addressing ghosting and texture loss that hinder traditional NeRF. It introduces TeX-NeRF, which preprocesses infrared inputs with Pseudo-TeX Vision to decompose radiance into temperature $T$, emissivity $e$, and texture $X$, then maps these to HSV channels and renders via an HSV-adapted NeRF. A physical radiance model $S_{ u}=e_{ u}B_{ u}(T)+[1-e_{ u}]X_{ u}$ plus Planck’s law underpins the decomposition, with a known emissivity prior $e(m)$ to resolve the inverse problem. Experiments on the 3D-TeX and ThermalMix datasets show TeX-NeRF achieves state-of-the-art novel-view synthesis quality and accurate temperature estimation, surpassing RGB-based baselines in challenging infrared conditions and enabling robust night-vision applications.

Abstract

Neural radiance fields (NeRF) has gained significant attention for its exceptional visual effects. However, most existing NeRF methods reconstruct 3D scenes from RGB images captured by visible light cameras. In practical scenarios like darkness, low light, or bad weather, visible light cameras become ineffective. Therefore, we propose TeX-NeRF, a 3D reconstruction method using only infrared images, which introduces the object material emissivity as a priori, preprocesses the infrared images using Pseudo-TeX vision, and maps the temperatures (T), emissivities (e), and textures (X) of the scene into the saturation (S), hue (H), and value (V) channels of the HSV color space, respectively. Novel view synthesis using the processed images has yielded excellent results. Additionally, we introduce 3D-TeX Datasets, the first dataset comprising infrared images and their corresponding Pseudo-TeX vision images. Experiments demonstrate that our method not only matches the quality of scene reconstruction achieved with high-quality RGB images but also provides accurate temperature estimations for objects in the scene.

Figures (4)

• Figure 1: Pseudo-TeX Vision schematic. Enables infrared images to overcome ghosting effects and recover texture detail information.
• Figure 2: Overview of the proposed approach. The infrared image undergoes blackbody correction to extract temperature distribution information. Material data for each object in the scene is obtained through thermal vision-based semantic segmentation, combined with emissivity a priori information. Texture details are extracted using enhancement algorithms like the sensor's AGC. This data is mapped to the HSV color space to generate Pseudo-TeX Vision, and the processed image sequences are used as inputs for training TeX-NeRF.
• Figure 3: Diagram of TeX-NeRF network architecture.
• Figure 4: Comparison of 3D scene reconstruction point clouds using TeX-NeRF and other methods on constant temperature heating table images processed by Pseudo-TeX Vision.