Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Real-Time Wildfire Localization on the NASA Autonomous Modular Sensor using Deep Learning

Yajvan Ravan, Aref Malek, Chester Dolph, Nikhil Behari

TL;DR

This work tackles real-time wildfire localization using high-altitude multispectral imagery by introducing a manually annotated 12-band AMS dataset and a two-tier neural network that first classifies patches and then segments fire regions. The approach leverages SWIR, IR, and thermal bands to achieve high accuracy and recall, including nighttime and cloud-occluded scenarios, with an IoU around $0.74$–$0.75$ and recall near $0.84$. Real-time feasibility is demonstrated via patch-based streaming, achieving over $100$ fps on a laptop GPU and enabling rapid, automated wildfire perimeter localization suitable for integration with existing incident-response pipelines. The dataset, code, and findings underscore the value of full-spectrum airborne imagery for robust wildfire detection and motivate future human-in-the-loop and pipeline-automation work."

Abstract

High-altitude, multi-spectral, aerial imagery is scarce and expensive to acquire, yet it is necessary for algorithmic advances and application of machine learning models to high-impact problems such as wildfire detection. We introduce a human-annotated dataset from the NASA Autonomous Modular Sensor (AMS) using 12-channel, medium to high altitude (3 - 50 km) aerial wildfire images similar to those used in current US wildfire missions. Our dataset combines spectral data from 12 different channels, including infrared (IR), short-wave IR (SWIR), and thermal. We take imagery from 20 wildfire missions and randomly sample small patches to generate over 4000 images with high variability, including occlusions by smoke/clouds, easily-confused false positives, and nighttime imagery. We demonstrate results from a deep-learning model to automate the human-intensive process of fire perimeter determination. We train two deep neural networks, one for image classification and the other for pixel-level segmentation. The networks are combined into a unique real-time segmentation model to efficiently localize active wildfire on an incoming image feed. Our model achieves 96% classification accuracy, 74% Intersection-over-Union(IoU), and 84% recall surpassing past methods, including models trained on satellite data and classical color-rule algorithms. By leveraging a multi-spectral dataset, our model is able to detect active wildfire at nighttime and behind clouds, while distinguishing between false positives. We find that data from the SWIR, IR, and thermal bands is the most important to distinguish fire perimeters. Our code and dataset can be found here: https://github.com/nasa/Autonomous-Modular-Sensor-Wildfire-Segmentation/tree/main and https://drive.google.com/drive/folders/1-u4vs9rqwkwgdeeeoUhftCxrfe_4QPTn?=usp=drive_link

Real-Time Wildfire Localization on the NASA Autonomous Modular Sensor using Deep Learning

TL;DR

This work tackles real-time wildfire localization using high-altitude multispectral imagery by introducing a manually annotated 12-band AMS dataset and a two-tier neural network that first classifies patches and then segments fire regions. The approach leverages SWIR, IR, and thermal bands to achieve high accuracy and recall, including nighttime and cloud-occluded scenarios, with an IoU around – and recall near . Real-time feasibility is demonstrated via patch-based streaming, achieving over fps on a laptop GPU and enabling rapid, automated wildfire perimeter localization suitable for integration with existing incident-response pipelines. The dataset, code, and findings underscore the value of full-spectrum airborne imagery for robust wildfire detection and motivate future human-in-the-loop and pipeline-automation work."

Abstract

High-altitude, multi-spectral, aerial imagery is scarce and expensive to acquire, yet it is necessary for algorithmic advances and application of machine learning models to high-impact problems such as wildfire detection. We introduce a human-annotated dataset from the NASA Autonomous Modular Sensor (AMS) using 12-channel, medium to high altitude (3 - 50 km) aerial wildfire images similar to those used in current US wildfire missions. Our dataset combines spectral data from 12 different channels, including infrared (IR), short-wave IR (SWIR), and thermal. We take imagery from 20 wildfire missions and randomly sample small patches to generate over 4000 images with high variability, including occlusions by smoke/clouds, easily-confused false positives, and nighttime imagery. We demonstrate results from a deep-learning model to automate the human-intensive process of fire perimeter determination. We train two deep neural networks, one for image classification and the other for pixel-level segmentation. The networks are combined into a unique real-time segmentation model to efficiently localize active wildfire on an incoming image feed. Our model achieves 96% classification accuracy, 74% Intersection-over-Union(IoU), and 84% recall surpassing past methods, including models trained on satellite data and classical color-rule algorithms. By leveraging a multi-spectral dataset, our model is able to detect active wildfire at nighttime and behind clouds, while distinguishing between false positives. We find that data from the SWIR, IR, and thermal bands is the most important to distinguish fire perimeters. Our code and dataset can be found here: https://github.com/nasa/Autonomous-Modular-Sensor-Wildfire-Segmentation/tree/main and https://drive.google.com/drive/folders/1-u4vs9rqwkwgdeeeoUhftCxrfe_4QPTn?=usp=drive_link
Paper Structure (16 sections, 1 equation, 5 figures, 5 tables)

This paper contains 16 sections, 1 equation, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets
  4. Image Segmentation
  5. Color-Rule Algorithms
  6. Deep Learning Approaches to Wildfire Detection
  7. Methodology
  8. Dataset Preparation
  9. Networks
  10. Training and Evaluation
  11. Two-Tier Model
  12. Results and Discussion
  13. Real-Time Simulation
  14. Spectral Ablation
  15. Conclusion
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Algorithm for identifying unambiguous fire pixels according to Schroeder et al.SchroederPaper
  • Figure 2: Example of Different AMS Spectral Bands From Our Dataset. (a) Bands Red/Green/Blue, (b) Bands 7/Red/Green, (c) Bands SWIR/7/Red, (d) Band 12 (Thermal), (e) Band 11 (IR), (f) Band 10 (SWIR). Active fire spots are clearly visible in c-f, but obscured in a-b.
  • Figure 3: Demonstration of patching on AMS dataset
  • Figure 4: Inference model combining classification and segmentation networks. Classification detects fire presence; segmentation localizes fire only when needed.
  • Figure 5: Example of Results.