Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Seeing Heat with Color -- RGB-Only Wildfire Temperature Inference from SAM-Guided Multimodal Distillation using Radiometric Ground Truth

Michael Marinaccio, Fatemeh Afghah

TL;DR

SAM-TIFF presents a multimodal-to-unimodal distillation framework that enables per-pixel wildfire temperature regression using RGB UAV imagery by distilling knowledge from a RGB–Thermal teacher trained on radiometric TIFF ground truth. The approach combines SAM-guided pseudo-labeling, Canny/Otsu edge cues, and TOPSIS-based mask selection to generate high-quality supervision for an RGB-only student tasked with both fire segmentation and temperature prediction. Pretraining on FLAME 2 and evaluation on FLAME 3 demonstrate robust segmentation and temperature estimation, achieving a mean IoU of up to 0.714 and temperature accuracy within $±$50°C for a substantial portion of fire pixels. This work enables cost-effective, thermal-sensor-free wildfire monitoring with near-modal performance, potentially expanding deployment to lightweight UAVs and consumer devices while maintaining temperature-aware situational awareness.

Abstract

High-fidelity wildfire monitoring using Unmanned Aerial Vehicles (UAVs) typically requires multimodal sensing - especially RGB and thermal imagery - which increases hardware cost and power consumption. This paper introduces SAM-TIFF, a novel teacher-student distillation framework for pixel-level wildfire temperature prediction and segmentation using RGB input only. A multimodal teacher network trained on paired RGB-Thermal imagery and radiometric TIFF ground truth distills knowledge to a unimodal RGB student network, enabling thermal-sensor-free inference. Segmentation supervision is generated using a hybrid approach of segment anything (SAM)-guided mask generation, and selection via TOPSIS, along with Canny edge detection and Otsu's thresholding pipeline for automatic point prompt selection. Our method is the first to perform per-pixel temperature regression from RGB UAV data, demonstrating strong generalization on the recent FLAME 3 dataset. This work lays the foundation for lightweight, cost-effective UAV-based wildfire monitoring systems without thermal sensors.

Seeing Heat with Color -- RGB-Only Wildfire Temperature Inference from SAM-Guided Multimodal Distillation using Radiometric Ground Truth

TL;DR

SAM-TIFF presents a multimodal-to-unimodal distillation framework that enables per-pixel wildfire temperature regression using RGB UAV imagery by distilling knowledge from a RGB–Thermal teacher trained on radiometric TIFF ground truth. The approach combines SAM-guided pseudo-labeling, Canny/Otsu edge cues, and TOPSIS-based mask selection to generate high-quality supervision for an RGB-only student tasked with both fire segmentation and temperature prediction. Pretraining on FLAME 2 and evaluation on FLAME 3 demonstrate robust segmentation and temperature estimation, achieving a mean IoU of up to 0.714 and temperature accuracy within 50°C for a substantial portion of fire pixels. This work enables cost-effective, thermal-sensor-free wildfire monitoring with near-modal performance, potentially expanding deployment to lightweight UAVs and consumer devices while maintaining temperature-aware situational awareness.

Abstract

High-fidelity wildfire monitoring using Unmanned Aerial Vehicles (UAVs) typically requires multimodal sensing - especially RGB and thermal imagery - which increases hardware cost and power consumption. This paper introduces SAM-TIFF, a novel teacher-student distillation framework for pixel-level wildfire temperature prediction and segmentation using RGB input only. A multimodal teacher network trained on paired RGB-Thermal imagery and radiometric TIFF ground truth distills knowledge to a unimodal RGB student network, enabling thermal-sensor-free inference. Segmentation supervision is generated using a hybrid approach of segment anything (SAM)-guided mask generation, and selection via TOPSIS, along with Canny edge detection and Otsu's thresholding pipeline for automatic point prompt selection. Our method is the first to perform per-pixel temperature regression from RGB UAV data, demonstrating strong generalization on the recent FLAME 3 dataset. This work lays the foundation for lightweight, cost-effective UAV-based wildfire monitoring systems without thermal sensors.
Paper Structure (13 sections, 4 figures, 4 tables)

This paper contains 13 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. UAV-Based Wildfire Monitoring Datasets
  3. FLAME 2 Dataset
  4. FLAME 3 Dataset
  5. Thermal TIFF Analysis and Calibration
  6. Related Work
  7. SAM-TIFF Temperature Inference
  8. Hybrid Mask Generation
  9. Positive and Negative Point Autolocator
  10. Final Ground Truth Mask Decision
  11. FLAME 2 Pretraining
  12. Results
  13. Conclusions and Future Work

Figures (4)

  • Figure 1: TIFF Temperature Range Visualization - Sycan Marsh Sample; Temperature Graph (Left), Thermal JPG (Right)
  • Figure 2: SAM-TIFF Architecture (a), Autpoint Locator Algorithm (b) - 'TIFF Guided Point Prompts' Module in (a); Jet Colormap for TIFF Visualization, Inferno Colormap for Thermal JPG Visualization; Canny Edge Detection Thresholds - Low Threshold = $\tau$ and High Threshold = 200 degrees Celsius; $\epsilon$ = 25 degrees Celsius
  • Figure 3: Segmentation predictions for SFAFMA-50 Teacher - DeepLabV3+ Student Variant
  • Figure 4: Willamette Valley Sample Image Results for Teacher Networks (left) - DeepLabV3+ Student Variants - Temp Predictions [Jet Colormap] between 0 and 500 ($^\circ$C)