Drone Detection using Deep Neural Networks Trained on Pure Synthetic Data

TL;DR

The results show that using synthetic data for drone detection has the potential to reduce data collection costs and improve labelling quality, and may enable reliable drone detection systems, which could benefit other areas, such as unmanned traffic management systems.

Abstract

Drone detection has benefited from improvements in deep neural networks, but like many other applications, suffers from the availability of accurate data for training. Synthetic data provides a potential for low-cost data generation and has been shown to improve data availability and quality. However, models trained on synthetic datasets need to prove their ability to perform on real-world data, known as the problem of sim-to-real transferability. Here, we present a drone detection Faster-RCNN model trained on a purely synthetic dataset that transfers to real-world data. We found that it achieves an AP_50 of 97.0% when evaluated on the MAV-Vid - a real dataset of flying drones - compared with 97.8% for an equivalent model trained on real-world data. Our results show that using synthetic data for drone detection has the potential to reduce data collection costs and improve labelling quality. These findings could be a starting point for more elaborate synthetic drone datasets. For example, realistic recreations of specific scenarios could de-risk the dataset generation of safety-critical applications such as the detection of drones at airports. Further, synthetic data may enable reliable drone detection systems, which could benefit other areas, such as unmanned traffic management systems. The code is available https://github.com/mazqtpopx/cranfield-synthetic-drone-detection alongside the datasets https://huggingface.co/datasets/mazqtpopx/cranfield-synthetic-drone-detection.

Figures (8)

• Figure 1: The experimental process is split into 3 parts: synthetic dataset generation, training of neural networks (NNs), and testing. During the synthetic dataset generation, the position of the camera and HDRIs (background and lighting setup) are randomized. 4 distinct synthetic datasets are created: drones only, drones and birds, drones and generic distractors, drones and detailed distractors. A Faster-RCNN model is trained on the synthetic datasets. The Faster-RCNN model is then tested on each of the real-world datasets: MAV-VID, Drone-vs-Bird, and Anti-UAV.
• Figure 2: Synthetic datasets generated by using different styles. (a) Drones only, (b) drones and birds, (c) generic distractors, (d) realistic distractors, (e) random backgrounds.
• Figure 3: A zoomed-in example of an inaccurate ground truth on the MAV-Vid dataset. Although the model prediction (blue rectangle) is more accurate than the ground truth (white rectangle), it falls below the intersection over union (IoU) of 0.5 required for a correct prediction.
• Figure 4: A representation selected bounds within which the camera position is randomized. Red: 40 m, blue: 80 m, green: 160 m. Each frame, the position of the camera is randomized within these bounds. We also present results from 20 m and 320 m bounds, which are not illustrated here.
• Figure 5: Results of the bounding size study for 20 m, 40 m, 80 m, 160 m, and 320 m bounds. Average Precision at .50 IoU ($AP_{0.5}$) on the y-axis, presented as a mean of 8 runs along with a 95 percent confidence interval, and the selected bounding size on the x-axis. Each bounding size is tested on the MAV-Vid, Drone-vs-Bird (DvB), and Anti-UAV datasets.