Vision-only UAV State Estimation for Fast Flights Without External Localization Systems: A2RL Drone Racing Finalist Approach

TL;DR

This work tackles GNSS-denied, high-speed UAV state estimation using only a monocular camera and an IMU. It introduces a modular fusion architecture that combines VIO, an onboard landmark-based gate detector, and an IMU-driven drift model, with a Linear Kalman Filter correcting translational and rotational drift to produce full $6$-DOF estimates at $100$ Hz. The approach achieves substantial accuracy gains over prior VIO-based methods (e.g., $70\%$ RMSE reduction in orientation, $16\%$ in linear velocity, and $8\times$ in angular velocity) and demonstrates robust performance in simulations, outdoor RTK-ground-truth experiments (RMSE reduction by up to $27\times$ for position), and competitive results in the A2RL Drone Racing Challenge 2025, where the team reached the finals. By decomposing sensing, drift correction, and control with open-source availability, the method offers a practical, industrially relevant solution for fast, vision-only UAV navigation in cluttered GNSS-denied environments.

Abstract

Fast flights with aggressive maneuvers in cluttered GNSS-denied environments require fast, reliable, and accurate UAV state estimation. In this paper, we present an approach for onboard state estimation of a high-speed UAV using a monocular RGB camera and an IMU. Our approach fuses data from Visual-Inertial Odometry (VIO), an onboard landmark-based camera measurement system, and an IMU to produce an accurate state estimate. Using onboard measurement data, we estimate and compensate for VIO drift through a novel mathematical drift model. State-of-the-art approaches often rely on more complex hardware (e.g., stereo cameras or rangefinders) and use uncorrected drifting VIO velocities, orientation, and angular rates, leading to errors during fast maneuvers. In contrast, our method corrects all VIO states (position, orientation, linear and angular velocity), resulting in accurate state estimation even during rapid and dynamic motion. Our approach was thoroughly validated through 1600 simulations and numerous real-world experiments. Furthermore, we applied the proposed method in the A2RL Drone Racing Challenge 2025, where our team advanced to the final four out of 210 teams and earned a medal.

Figures (7)

• Figure 1: Long-exposure photo of drones carrying blue lights taken during a race flight at the A2RL Drone Racing Challenge 2025.
• Figure 2: This figure presents the pipeline diagram of the estimation approach proposed in this paper, highlighted by the light blue dotted rectangle and integrated into our drone racing framework. The High-level mission planning block provides the desired position and heading references $(\mathbf{r}_d,\eta_d)$ for the UAV to follow the race track. These references are passed to the Reference tracker, which generates a smooth and dynamically feasible reference $\bm{\chi}_d$ for the Reference controller. The Reference controller computes the desired thrust and angular velocities $(\bm{\omega}_d,~T_d)$ for the embedded flight controller, which then sends actuator commands $\bm{\tau}_{cm}$ to the UAV actuators. The VIO algorithm uses IMU data and onboard camera images to estimate the UAV position $\mathbf{x}$, orientation $\mathbf{R}$, linear velocity $\dot{\mathbf{x}}$, and angular velocity $\bm{\omega}$. The VIO drift estimator estimates the drift in the VIO states using position $\mathbf{x}$ and orientation $\mathbf{R}$ measurements from the Landmark detector. The State estimator fuses data from the IMU, VIO drift estimator, and VIO algorithm to estimate the UAV position, orientation, and linear and angular velocities $(\mathbf{x},\dot{\mathbf{x}},\mathbf{R},\bm{\omega})$.
• Figure 3: The depiction of coordinate frames consisting of the world frame $\mathcal{W}{}$, UAV body frame $\mathcal{B}_{w}{}$ expressed in the $\mathcal{W}{}$ frame, VIO frame $\mathcal{V}{}$, and UAV body frame $\mathcal{B}_{v}{}$ expressed in the $\mathcal{V}{}$ frame. The relative position and orientation between the $\mathcal{W}{}$ frame and the $\mathcal{V}{}$ frame are represented by $\mathbf{r}_{\text{init}}^w$. The UAV position and orientation in $\mathcal{W}{}$ frame are denoted as $\mathbf{r}_{\text{gt}}^w$. The UAV position and orientation estimated by the VIO algorithm are provided in the $\mathcal{V}{}$ frame and denoted as $\mathbf{r}_{\text{vio}}^v$. The vector $\mathbf{r}_{\text{vio}}^w$ represents the VIO-estimated UAV position and orientation expressed in the $\mathcal{W}{}$ frame. The VIO drift is expressed as $\mathbf{r}_{\text{drift}}^w$.
• Figure 4: Custom-built outdoor track used to verify our approach.
• Figure 5: Top view of the outdoor track showing the RTK states, results of our approach, VIO states, and data from the Landmark detector (LM det.).