Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing

TL;DR

This work tackles the challenge of reliable, high-rate ground localization for safe drone landings in cluttered or GPS-degraded environments. It introduces mmE-Loc, a two-module system that couples an event camera with mmWave radar to achieve cm-scale accuracy and ms-scale latency; Consistency-Instructed Collaborative Tracking (CCT) filters noise and provides preliminary drone measurements, while Graph-Informed Adaptive Joint Optimization (GAJO) fuses modalities via a factor-graph framework with adaptive optimization. The main contributions are the sensor configuration (event camera + mmWave radar), the CCT and GAJO algorithms, and extensive real-world validation showing superior accuracy and latency compared to state-of-the-art baselines, including deployment at a real drone airport. The results demonstrate that mmE-Loc can robustly support safe landings under varying illumination, occlusion, and background dynamics, making it a practical complement to RTK and visual markers in commercial drone operations.

Abstract

For precise, efficient, and safe drone landings, ground platforms should real-time, accurately locate descending drones and guide them to designated spots. While mmWave sensing combined with cameras improves localization accuracy, the lower sampling frequency of traditional frame cameras compared to mmWave radar creates bottlenecks in system throughput. In this work, we replace the traditional frame camera with event camera, a novel sensor that harmonizes in sampling frequency with mmWave radar within the ground platform setup, and introduce mmE-Loc, a high-precision, low-latency ground localization system designed for drone landings. To fully leverage the \textit{temporal consistency} and \textit{spatial complementarity} between these modalities, we propose two innovative modules, \textit{consistency-instructed collaborative tracking} and \textit{graph-informed adaptive joint optimization}, for accurate drone measurement extraction and efficient sensor fusion. Extensive real-world experiments in landing scenarios from a leading drone delivery company demonstrate that mmE-Loc outperforms state-of-the-art methods in both localization accuracy and latency.

Figures (23)

• Figure 1: Snapshot of drone landing phase, airport, and sensors performance. (a) A delivery drone lands on the platform. (b) The real-world drone airport is equipped with multiple drones for package delivery. (c) Integrating mmWave radar with event camera combines reliable depth sensing and 2D imaging at ultra-high sampling frequencies, enabling high spatial-temporal resolution and depth sensing, while ensuring compatibility with flight controllers.
• Figure 2: Benchmark study on drone localization. (a) Benchmark study at a real-world drone delivery airport; (b) Both sensors are sensitive to environmental variations; (c) Existing algorithms suffer from low noise filtering rates; (d) Existing algorithms experience cumulative drift errors and delays.
• Figure 3: System architecture of mmE-Loc.
• Figure 4: Illustration of tracking models in Consistency-instructed Collaborative Tracking algorithm.
• Figure 5: Illustration of synchronous frames and asynchronous events. Frame cameras use a global shutter to capture images at fixed intervals, while each pixel in an event camera responds independently, generating events asynchronously when intensity changes exceed a threshold.