Thermoxels: a voxel-based method to generate simulation-ready 3D thermal models

TL;DR

This work targets data-driven retrofit of buildings by providing a voxel-based method that produces simulation-ready RGB+thermal 3D models for FEA. Thermoxels jointly optimizes geometry and temperature on a voxel grid with per-voxel density $\sigma$, color via spherical harmonics $SH$, and temperature $T$, using differentiable rendering to align synthetic views with sparse RGB and thermal images. After optimization, a densest-voxel extraction yields a connected, FEM-friendly volumetric mesh, which can be used in simple heat-conduction simulations (e.g., via $\text{JaxFEM}$) starting from the estimated thermal field. Compared to RGB-only or thermal-only baselines, Thermoxels emphasizes geometry fidelity while obtaining usable temperature distributions, enabling practical, data-driven retrofit assessments, though it requires further work on robustly handling reflective/flat surfaces and incorporating realistic material properties.

Abstract

In the European Union, buildings account for 42% of energy use and 35% of greenhouse gas emissions. Since most existing buildings will still be in use by 2050, retrofitting is crucial for emissions reduction. However, current building assessment methods rely mainly on qualitative thermal imaging, which limits data-driven decisions for energy savings. On the other hand, quantitative assessments using finite element analysis (FEA) offer precise insights but require manual CAD design, which is tedious and error-prone. Recent advances in 3D reconstruction, such as Neural Radiance Fields (NeRF) and Gaussian Splatting, enable precise 3D modeling from sparse images but lack clearly defined volumes and the interfaces between them needed for FEA. We propose Thermoxels, a novel voxel-based method able to generate FEA-compatible models, including both geometry and temperature, from a sparse set of RGB and thermal images. Using pairs of RGB and thermal images as input, Thermoxels represents a scene's geometry as a set of voxels comprising color and temperature information. After optimization, a simple process is used to transform Thermoxels' models into tetrahedral meshes compatible with FEA. We demonstrate Thermoxels' capability to generate RGB+Thermal meshes of 3D scenes, surpassing other state-of-the-art methods. To showcase the practical applications of Thermoxels' models, we conduct a simple heat conduction simulation using FEA, achieving convergence from an initial state defined by Thermoxels' thermal reconstruction. Additionally, we compare Thermoxels' image synthesis abilities with current state-of-the-art methods, showing competitive results, and discuss the limitations of existing metrics in assessing mesh quality.

Figures (2)

• Figure 1: Pipeline of the proposed Thermoxels framework transforming input images into a simulation-ready 3D thermal model. Given a set of RGB and thermal images, the density ($\sigma$), color (spherical harmonics $SH$), and temperature ($T$) of a voxel-based 3D model are optimized by comparing synthesized RGB and thermal images (based only on camera pose) against ground-truth measurements. Images below the voxel grid are Thermoxels-generated views on the Building A spring dataset.
• Figure 2: Examples of meshes reconstructed using Thermoxels and Plenoxels$_t$. One can see that Thermoxels consistently leads to more accurate reconstructions than Plenoxels$_t$. Due to the spherical harmonics used by Plenoxels$_t$, the temperature on the surface varies depending on the viewing direction, which is why the exported meshes do not have associated temperatures. Blue depicts the minimum temperature (Celsius) of the scene, grey average, and red maximum.