Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Stage Fusion Architecture for Small-Drone Localization and Identification Using Passive RF and EO Imagery: A Case Study

Thakshila Wimalajeewa Wewelwala, Thomas W. Tedesso, Tony Davis

TL;DR

A multi-stage fusion architecture using passive radio frequency (RF) and electro-optic imagery data is developed to leverage the synergies of the modalities to improve the overall tracking and classification capabilities of small drones.

Abstract

Reliable detection, localization and identification of small drones is essential to promote safe, secure and privacy-respecting operation of Unmanned-Aerial Systems (UAS), or simply, drones. This is an increasingly challenging problem with only single modality sensing, especially, to detect and identify small drones. In this work, a multi-stage fusion architecture using passive radio frequency (RF) and electro-optic (EO) imagery data is developed to leverage the synergies of the modalities to improve the overall tracking and classification capabilities. For detection with EO-imagery, supervised deep learning based techniques as well as unsupervised foreground/background separation techniques are explored to cope with challenging environments. Using real collected data for Group 1 and 2 drones, the capability of each algorithm is quantified. In order to compensate for any performance gaps in detection with only EO imagery as well as to provide a unique device identifier for the drones, passive RF is integrated with EO imagery whenever available. In particular, drone detections in the image plane are combined with passive RF location estimates via detection-to-detection association after 3D to 2D transformation. Final tracking is performed on the composite detections in the 2D image plane. Each track centroid is given a unique identification obtained via RF fingerprinting. The proposed fusion architecture is tested and the tracking and performance is quantified over the range to illustrate the effectiveness of the proposed approaches using simultaneously collected passive RF and EO data at the Air Force Research Laboratory (AFRL) through ESCAPE-21 (Experiments, Scenarios, Concept of Operations, and Prototype Engineering) data collect

Multi-Stage Fusion Architecture for Small-Drone Localization and Identification Using Passive RF and EO Imagery: A Case Study

TL;DR

A multi-stage fusion architecture using passive radio frequency (RF) and electro-optic imagery data is developed to leverage the synergies of the modalities to improve the overall tracking and classification capabilities of small drones.

Abstract

Reliable detection, localization and identification of small drones is essential to promote safe, secure and privacy-respecting operation of Unmanned-Aerial Systems (UAS), or simply, drones. This is an increasingly challenging problem with only single modality sensing, especially, to detect and identify small drones. In this work, a multi-stage fusion architecture using passive radio frequency (RF) and electro-optic (EO) imagery data is developed to leverage the synergies of the modalities to improve the overall tracking and classification capabilities. For detection with EO-imagery, supervised deep learning based techniques as well as unsupervised foreground/background separation techniques are explored to cope with challenging environments. Using real collected data for Group 1 and 2 drones, the capability of each algorithm is quantified. In order to compensate for any performance gaps in detection with only EO imagery as well as to provide a unique device identifier for the drones, passive RF is integrated with EO imagery whenever available. In particular, drone detections in the image plane are combined with passive RF location estimates via detection-to-detection association after 3D to 2D transformation. Final tracking is performed on the composite detections in the 2D image plane. Each track centroid is given a unique identification obtained via RF fingerprinting. The proposed fusion architecture is tested and the tracking and performance is quantified over the range to illustrate the effectiveness of the proposed approaches using simultaneously collected passive RF and EO data at the Air Force Research Laboratory (AFRL) through ESCAPE-21 (Experiments, Scenarios, Concept of Operations, and Prototype Engineering) data collect

Paper Structure

This paper contains 22 sections, 11 equations, 17 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Description of Data and Drones
  3. Problem Formulation/Solution Overview
  4. Drone Detection with EO Imagery
  5. Image Detection with Deep Learning
  6. Image Detection via Foreground/Background Separation
  7. Drone Detection, Localization and Identification with Passive RF
  8. Drone Identification via RF Fingerprinting
  9. Localization
  10. Detection-Detection Association and Tracking with EO and Passive RF in 2D
  11. Performance Analysis
  12. Analyzing r06
  13. r06: Drone Detection Performance with EO Imagery
  14. r06: Drone Classification Performance with RF Fingerprinting
  15. r06: TDOA Based 3D Localization
  16. ...and 7 more sections

Figures (17)

  • Figure 1: AFRL ESCAPE2021 data collect: Types of drones used as targets; picture credit: AFRL/RI [13]
  • Figure 2: AFRL ESCAPE2021 data collect: Sensor Layout with passive RF and EO Sensors; picture credit: AFRL/RI [13]
  • Figure 3: EO and Passive RF individual operations and the fusion architecture for detection, identification and tracking of drones
  • Figure 4: DETR concept for image detection [20]
  • Figure 5: Overview of RPCA for foreground/background separation with EO imagery
  • ...and 12 more figures