Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing

Maulana Bisyir Azhari, Donghun Han, SungJun Park, David Hyunchul Shim

TL;DR

This work proposes ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing and introduces ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments.

Abstract

Autonomous drone racing (ADR) demands state estimation that is simultaneously computationally efficient and resilient to the perceptual degradation experienced during extreme velocity and maneuvers. Traditional frameworks typically rely on conventional visual-inertial pipelines with loosely-coupled gate-based Perspective-n-Points (PnP) corrections that suffer from a rigid requirement for four visible features and information loss in intermediate steps. Furthermore, the absence of GNSS and Motion Capture systems in uninstrumented, competitive racing environments makes the objective evaluation of such systems remarkably difficult. To address these limitations, we propose ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing. Our approach integrates direct pixel reprojection errors from gate corners features as innovation terms within the filter. By bypassing intermediate PnP solvers, ADR-VINS maintains valid state updates with as few as two visible corners and utilizes robust reweighting instead of RANSAC-based schemes to handle outliers, enhancing computational efficiency. Furthermore, we introduce ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments. The proposed system is validated using TII-RATM dataset, where ADR-VINS achieves an average RMS translation error of 0.134 m, while ADR-FGO yields 0.060 m as a smoothing-based reference. Finally, ADR-VINS was successfully deployed in the A2RL Drone Championship Season 2, maintaining stable and robust estimation despite noisy detections during high-agility flight at top speeds of 20.9 m/s. We further utilize ADR-FGO for post-flight evaluation in uninstrumented racing environments.

Robust Tightly-Coupled Filter-Based Monocular Visual-Inertial State Estimation and Graph-Based Evaluation for Autonomous Drone Racing

TL;DR

This work proposes ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing and introduces ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments.

Abstract

Autonomous drone racing (ADR) demands state estimation that is simultaneously computationally efficient and resilient to the perceptual degradation experienced during extreme velocity and maneuvers. Traditional frameworks typically rely on conventional visual-inertial pipelines with loosely-coupled gate-based Perspective-n-Points (PnP) corrections that suffer from a rigid requirement for four visible features and information loss in intermediate steps. Furthermore, the absence of GNSS and Motion Capture systems in uninstrumented, competitive racing environments makes the objective evaluation of such systems remarkably difficult. To address these limitations, we propose ADR-VINS, a robust, monocular visual-inertial state estimation framework based on an Error-State Kalman Filter (ESKF) tailored for autonomous drone racing. Our approach integrates direct pixel reprojection errors from gate corners features as innovation terms within the filter. By bypassing intermediate PnP solvers, ADR-VINS maintains valid state updates with as few as two visible corners and utilizes robust reweighting instead of RANSAC-based schemes to handle outliers, enhancing computational efficiency. Furthermore, we introduce ADR-FGO, an offline Factor-Graph Optimization framework to generate high-fidelity reference trajectories that facilitate post-flight performance evaluation and analysis on uninstrumented, GNSS-denied environments. The proposed system is validated using TII-RATM dataset, where ADR-VINS achieves an average RMS translation error of 0.134 m, while ADR-FGO yields 0.060 m as a smoothing-based reference. Finally, ADR-VINS was successfully deployed in the A2RL Drone Championship Season 2, maintaining stable and robust estimation despite noisy detections during high-agility flight at top speeds of 20.9 m/s. We further utilize ADR-FGO for post-flight evaluation in uninstrumented racing environments.
Paper Structure (25 sections, 23 equations, 9 figures, 5 tables)

This paper contains 25 sections, 23 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. ADR-VINS
  4. Detection
  5. Reordering
  6. Association
  7. Error-State Kalman Filter
  8. State Definition
  9. IMU Propagation
  10. Update
  11. Post-Flight Evaluation via ADR-FGO
  12. Factor Formulations
  13. IMU Preintegration Factors
  14. Gate Corner Visual Factors
  15. ADR-VINS Prior Factors
  16. ...and 10 more sections

Figures (9)

  • Figure 1: Left: The autonomous drone racing platform used in the A2RL Drone Championship Season 2, featuring a low-cost monocular RGB camera and IMU sensors. Right: Autonomous racing flight at high speeds over 20 m/s featuring the proposed ADR-VINS for its real-time state estimation.
  • Figure 2: Overview of the proposed ADR-VINS. (a) The framework consists of a visual measurement pipeline (including detection, reordering, and association) and a tightly-coupled Error-State Kalman Filter (ESKF) that fuses raw IMU data with direct pixel reprojection errors to provide robust, high-rate state estimation. (b) Illustration of the corner detections (yellow dots), reprojections (red dots), and its errors (red lines).
  • Figure 3: The proposed factor graph structure in ADR-FGO. The graph integrates IMU pre-integration, gate corner detections, pose prior factors from ADR-VINS, and camera extrinsics priors. Keyframes are maintained even during visual outages to distribute drift and IMU biases across these visual-less sections to provide better interpolation.
  • Figure 4: Qualitative comparison of ADR-VINS against baseline online methods across various tracks on TII-RATM dataset autonomous sequences, including an ellipse (Flight 06A), a lemniscate (Flight 12A), and a complex 3D racing track (Flight 18A).
  • Figure 5: Comparison of ADR-VINS with $\#$2, $\#$4, and $\#$6 minimum corners visibility requirements for Flight 18A. Requiring only two corners enables earlier and more frequent updates, significantly reducing drift. ADR-FGO further minimizes this error by globally refining ADR-VINS, even in sections where gates are not visible. G1-G7 is the gate ordering.
  • ...and 4 more figures