Through-Foliage Surface-Temperature Reconstruction for early Wildfire Detection

TL;DR

This work tackles the challenge of detecting ground- and surface-level fires obscured by dense forest canopies by combining Airborne Optical Sectioning with a neural restoration pipeline. It leverages a hybrid data-generation approach—latent diffusion within a VQ-VAE framework plus procedural forest simulations—to produce large, diverse training sets that enable a visual state-space restoration model (VmambaIR) to recover ground temperatures from blurred AOS outputs. The method delivers substantial RMSE improvements over conventional thermal imaging and uncorrected AOS, both in simulations (2–2.5× reductions) and field experiments (up to 12.8× improvements for hotspots), while preserving the morphology of fire and human signatures. The approach runs in near real-time and is scalable to large forested areas via sliding-window AOS, with potential applicability to other thermal targets such as search-and-rescue indicators and to other wavelength bands in future work.

Abstract

We introduce a novel method for reconstructing surface temperatures through occluding forest vegetation by combining signal processing and machine learning. Our goal is to enable fully automated aerial wildfire monitoring using autonomous drones, allowing for the early detection of ground fires before smoke or flames are visible. While synthetic aperture (SA) sensing mitigates occlusion from the canopy and sunlight, it introduces thermal blur that obscures the actual surface temperatures. To address this, we train a visual state space model to recover the subtle thermal signals of partially occluded soil and fire hotspots from this blurred data. A key challenge was the scarcity of real-world training data. We overcome this by integrating a latent diffusion model into a vector quantized to generated a large volume of realistic surface temperature simulations from real wildfire recordings, which we further expanded through temperature augmentation and procedural thermal forest simulation. On simulated data across varied ambient and surface temperatures, forest densities, and sunlight conditions, our method reduced the RMSE by a factor of 2 to 2.5 compared to conventional thermal and uncorrected SA imaging. In field experiments focused on high-temperature hotspots, the improvement was even more significant, with a 12.8-fold RMSE gain over conventional thermal and a 2.6-fold gain over uncorrected SA images. We also demonstrate our model's generalization to other thermal signals, such as human signatures for search and rescue. Since simple thresholding is frequently inadequate for detecting subtle thermal signals, the morphological characteristics are equally essential for accurate classification. Our experiments demonstrated another clear advantage: we reconstructed the complete morphology of fire and human signatures, whereas conventional imaging is defeated by partial occlusion.

Figures (18)

• Figure 1: A georeferenced 3D reconstruction of the Grafenberg Alm wildfire site, alongside a sample of synchronized RGB and thermal drone images. The 543 individual drone capture poses are marked. Temperature statistics of the dataset and specifications of the thermal sensor are also provided.
• Figure 1: Evolution of the loss functions (here: L1, L2, perceptual loss, and Mean Squared Error, MSE) during the training of our VQ-VAE/LDM model, illustrating stable convergence behavior. The Fréchet Inception Distance (FID) score further demonstrates the high fidelity and perceptual quality of the generated thermal images.
• Figure 2: Model architecture for forest surface temperature generation. Recorded thermal image is first encoded by a VQ-VAE into a latent space with reduced spatial dimensionality, which serves as the input to the LDM. The LDM operates within this compact representation to model the underlying data distribution, after which the latent features are decoded to reconstruct the corresponding high-resolution image. The output is generated thermal image with realistic spatial and thermal characteristics.
• Figure 2: The sigmoid function $\psi(t)$ used in Eqn. 1 of the main manuscript to smoothen discontinuities at the boundaries of $T_{lower}$ and $T_{upper}$ (here, for example, $0^\circ C$ and $30^\circ C$) . The parameter $\alpha$ which controls smoothness of the temperature augmentation around lower and upper temperature thresholds. Empirically, we chose $\alpha$=0.5.
• Figure 3: A comparison of surface temperatures generated by our VQ-VAE/LDM model (left) against real drone recordings from the Grafenberg Alm wildfire (right). Temperatures are color-coded according to the scale. Note, that the generated data closely matches the recorded data in both the appearance and temperatures of ground structures and fire hotspots.