Real-Time Drone Detection in Event Cameras via Per-Pixel Frequency Analysis

TL;DR

This work proposes a novel per-pixel temporal analysis framework using the Non-uniform Discrete Fourier Transform (NDFT) to identify the frequency signature of drone rotors, as characterized by frequency combs in their power spectra, enabling a tunable and generalizable algorithm that achieves accurate real-time localization of UAV.

Abstract

Detecting fast-moving objects, such as unmanned aerial vehicle (UAV), from event camera data is challenging due to the sparse, asynchronous nature of the input. Traditional Discrete Fourier Transforms (DFT) are effective at identifying periodic signals, such as spinning rotors, but they assume uniformly sampled data, which event cameras do not provide. We propose a novel per-pixel temporal analysis framework using the Non-uniform Discrete Fourier Transform (NDFT), which we call Drone Detection via Harmonic Fingerprinting (DDHF). Our method uses purely analytical techniques that identify the frequency signature of drone rotors, as characterized by frequency combs in their power spectra, enabling a tunable and generalizable algorithm that achieves accurate real-time localization of UAV. We compare against a YOLO detector under equivalent conditions, demonstrating improvement in accuracy and latency across a difficult array of drone speeds, distances, and scenarios. DDHF achieves an average localization F1 score of 90.89% and average latency of 2.39ms per frame, while YOLO achieves an F1 score of 66.74% and requires 12.40ms per frame. Through utilization of purely analytic techniques, DDHF is quickly tuned on small data, easily interpretable, and achieves competitive accuracies and latencies to deep learning alternatives.

Figures (4)

• Figure 1: (a) Far distance drone test at 300m using the Navitar 100mm F2.8 lens. (b) Direct sunlight test demonstrating the wide dynamic range of event based vision. (c) Multi-drone test - both identified correctly. (d)-(e) Handheld shaky-cam examples. Algorithm is able to accurately isolate the drone from the background, while keeping latency to real-time speeds despite large event throughput. (f)-(h) Angle of elevation experiment at $0^\circ$, $10^\circ$, and $90^\circ$ respectively. At $0^\circ$, drone is barely visible (due to pose and possibly the focus of lens) and not identified, but at $10^\circ$, drone is accurately identified, showing a limitation of DDHF. At $90^\circ$ angle of elevation, the drone is most easily identified. (i) Power spectrum of pixel where two overlapping rotor paths projected onto. (j) Typical power spectrum of a single-rotor pixel.
• Figure 2: Visualization of one frame, comparing the spectra of a drone pixel versus a car wheel pixel. Both spectral flatness and harmonic comb scores from DDHF are able to distinguish between the spectrum of a spinning tire versus a spinning rotor.
• Figure 3: Demonstration of the interpretable tuning capabilities of DDHF. For row (a), the default hyperparameters obtained from the Bayesian optimization were used. For row (b), $\tau_{sf}$ was tuned from $0.89$ to $0.5$. For row (c), $\tau_{\mathrm{comb}}$ was tuned from $1.5$ to $6$. In both tuning cases, the bounding box becomes far more selective, filtering out the power spectra of the moving cars and effectively isolating the drone. Interestingly, only the inner parts of the drone rotor at this pose appear to be identified.
• Figure 4: Performance versus angle of elevation. (a) Accuracy measured by F1-score; (b) inference latency.