On Your Own: Pro-level Autonomous Drone Racing in Uninstrumented Arenas

TL;DR

This work demonstrates pro-level autonomous drone racing in both instrumented and uninstrumented arenas by delivering a perception–estimation–control stack that does not require ground-truth fine-tuning for drift correction. The system integrates a vision stack with gate detection, a VIO-based state estimator augmented by gate-PnP corrections, and a time-optimal trajectory generator coupled with model-predictive control, achieving high speeds and robust performance. It also publicly releases a substantial instrumented-track dataset and multisession results from a world-class pilot, enabling direct comparisons and benchmarking. The findings show that autonomous drones can outperform professional pilots in instrumented settings and remain competitive in uninstrumented environments, highlighting practical potential and guiding future work toward monocular sensing and safer, scalable deployment.

Abstract

Drone technology is proliferating in many industries, including agriculture, logistics, defense, infrastructure, and environmental monitoring. Vision-based autonomy is one of its key enablers, particularly for real-world applications. This is essential for operating in novel, unstructured environments where traditional navigation methods may be unavailable. Autonomous drone racing has become the de facto benchmark for such systems. State-of-the-art research has shown that autonomous systems can surpass human-level performance in racing arenas. However, direct applicability to commercial and field operations is still limited as current systems are often trained and evaluated in highly controlled environments. In our contribution, the system's capabilities are analyzed within a controlled environment -- where external tracking is available for ground-truth comparison -- but also demonstrated in a challenging, uninstrumented environment -- where ground-truth measurements were never available. We show that our approach can match the performance of professional human pilots in both scenarios. We also publicly release the data from the flights carried out by our approach and a world-class human pilot.

Figures (9)

• Figure 1: The two arenas used during the experiments, showing the autonomous drone (red) performing a lap in the instrumented one (top), and a head-to-head versus a human pilot (green) in the uninstrumented track (bottom).
• Figure 2: Quadrotors used in the experiments: the autonomous drone (including markers for the instrumented track, top); and the piloted drone (with replicas of the autonomy components, without markers for the uninstrumented track, bottom).
• Figure 3: Architecture diagram illustrating the data flow between the main hardware and software components, including: vision (green, Sec. \ref{['sec:vision-stack']}); state estimation (orange, Sec. \ref{['sec:state-estimation-stack']}); control (blue, Sec. \ref{['sec:control-stack']}); and autopilot (red, Sec. \ref{['sec:software:autopilot']}). Unlike scaramuzza-nature, we introduce a second-stage EKF leveraging the high-frequency FC IMU, which produces a sufficiently smooth state estimate to enable the use of classical control and MPC. High-frequency readings (500 Hz) from Betaflight are supported by our optimized MSP setup (Sec. \ref{['sec:software']}), offering higher performance than the SBUS protocol adopted in foehnscirob.
• Figure 4: Vision stack schematics illustrating the training procedure (top) and the detection pipeline (bottom). The training procedure comprises three steps (designed to minimize human supervision while preserving accuracy): (i) training on the RATM dataset bosello2024; (ii) auto-labeling, followed by human refinement in CVAT; and (iii) labeling track images with Grounding DINO from human examples. By the final stage, fine-tuning for peak performance in new environments requires as few as 80 human-corrected frames. In the detection pipeline, YOLOv8n YoloV8 detects gate bounding boxes, while MobileNetV3-Small mobilenetv3 estimates their keypoints. Leveraging GPU acceleration (see Sec. \ref{['sec:vision-stack']}), the stack achieves per-frame latency under 30 ms on $640\times640$ images. This yields lower delay and larger input resolution compared to scaramuzza-nature, resulting in higher pixel-level precision and potentially improved PnP accuracy.
• Figure 5: Gate setup for the instrumented (Sec. \ref{['sec:instrumented-track']}, top) and uninstrumented (Sec. \ref{['sec:uninstrumented-track']}, bottom) tracks, including starting point (in red), and the time-optimal reference trajectories (in blue). Yaw is reported for gates with non-zero yaw.