Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UAV-Assisted Real-Time Disaster Detection Using Optimized Transformer Model

Branislava Jankovic, Sabina Jangirova, Waseem Ullah, Latif U. Khan, Mohsen Guizani

TL;DR

The paper addresses real-time, privacy-preserving disaster detection from UAV imagery on resource-limited onboard platforms. It proposes an optimized Vision Transformer (Swin Transformer) with post-training quantization using TensorRT to meet edge constraints. Key contributions include a UAV-assisted edge framework, the DisasterEye eight-class dataset, and a comprehensive PTQ analysis demonstrating real-time edge inference with minimal accuracy loss on devices like the Jetson Nano. Findings show that FP16/INT8 TensorRT quantization enables scalable, real-time disaster monitoring on UAVs, with strong performance across diverse datasets and potential for deployment in real-world scenarios.

Abstract

Dangerous surroundings and difficult-to-reach landscapes introduce significant complications for adequate disaster management and recuperation. These problems can be solved by engaging unmanned aerial vehicles (UAVs) provided with embedded platforms and optical sensors. In this work, we focus on enabling onboard aerial image processing to ensure proper and real-time disaster detection. Such a setting usually causes challenges due to the limited hardware resources of UAVs. However, privacy, connectivity, and latency issues can be avoided. We suggest a UAV-assisted edge framework for disaster detection, leveraging our proposed model optimized for onboard real-time aerial image classification. The optimization of the model is achieved using post-training quantization techniques. To address the limited number of disaster cases in existing benchmark datasets and therefore ensure real-world adoption of our model, we construct a novel dataset, DisasterEye, featuring disaster scenes captured by UAVs and individuals on-site. Experimental results reveal the efficacy of our model, reaching high accuracy with lowered inference latency and memory use on both traditional machines and resource-limited devices. This shows that the scalability and adaptability of our method make it a powerful solution for real-time disaster management on resource-constrained UAV platforms.

UAV-Assisted Real-Time Disaster Detection Using Optimized Transformer Model

TL;DR

The paper addresses real-time, privacy-preserving disaster detection from UAV imagery on resource-limited onboard platforms. It proposes an optimized Vision Transformer (Swin Transformer) with post-training quantization using TensorRT to meet edge constraints. Key contributions include a UAV-assisted edge framework, the DisasterEye eight-class dataset, and a comprehensive PTQ analysis demonstrating real-time edge inference with minimal accuracy loss on devices like the Jetson Nano. Findings show that FP16/INT8 TensorRT quantization enables scalable, real-time disaster monitoring on UAVs, with strong performance across diverse datasets and potential for deployment in real-world scenarios.

Abstract

Dangerous surroundings and difficult-to-reach landscapes introduce significant complications for adequate disaster management and recuperation. These problems can be solved by engaging unmanned aerial vehicles (UAVs) provided with embedded platforms and optical sensors. In this work, we focus on enabling onboard aerial image processing to ensure proper and real-time disaster detection. Such a setting usually causes challenges due to the limited hardware resources of UAVs. However, privacy, connectivity, and latency issues can be avoided. We suggest a UAV-assisted edge framework for disaster detection, leveraging our proposed model optimized for onboard real-time aerial image classification. The optimization of the model is achieved using post-training quantization techniques. To address the limited number of disaster cases in existing benchmark datasets and therefore ensure real-world adoption of our model, we construct a novel dataset, DisasterEye, featuring disaster scenes captured by UAVs and individuals on-site. Experimental results reveal the efficacy of our model, reaching high accuracy with lowered inference latency and memory use on both traditional machines and resource-limited devices. This shows that the scalability and adaptability of our method make it a powerful solution for real-time disaster management on resource-constrained UAV platforms.
Paper Structure (18 sections, 4 figures, 2 tables)

This paper contains 18 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related work
  3. Vision Transformers
  4. Optimization Techniques
  5. Methodology
  6. Problem Formulation
  7. Model Architecture
  8. Model Optimization
  9. Model Inference
  10. Experimental Results
  11. Datasets
  12. Evaluation Metrics
  13. Training Performance
  14. Optimization Performance
  15. Visual Results Analysis
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Overview of the proposed framework: 1) Model training, where UAV-captured images are processed through preprocessing, feature extraction using a backbone transformer model, and training; and 2) Model inference, enabling real-time disaster detection.
  • Figure 2: Samples images of various UAVs based datasets: a) DFAN; b) AIDER; c) DisasterEye.
  • Figure 3: Class statistics for the DisasterEye dataset.
  • Figure 4: Visual Results of the proposed model on benchmark datasets: a) AIDER, b) DFAN, and c) DisasterEye.