Perception-Aware Time-Optimal Planning for Quadrotor Waypoint Flight

TL;DR

A unified time-optimal trajectory optimization framework for vision-based quadrotors that explicitly incorporates perception constraints alongside full nonlinear dynamics, rotor actuation limits, aerodynamic effects, camera field-of-view constraints, and convex geometric gate representations is introduced.

Abstract

Agile quadrotor flight pushes the limits of control, actuation, and onboard perception. While time-optimal trajectory planning has been extensively studied, existing approaches typically neglect the tight coupling between vehicle dynamics, environmental geometry, and the visual requirements of onboard state estimation. As a result, trajectories that are dynamically feasible may fail in closed-loop execution due to degraded visual quality. This paper introduces a unified time-optimal trajectory optimization framework for vision-based quadrotors that explicitly incorporates perception constraints alongside full nonlinear dynamics, rotor actuation limits, aerodynamic effects, camera field-of-view constraints, and convex geometric gate representations. The proposed formulation solves minimum-time lap trajectories for arbitrary racetracks with diverse gate shapes and orientations, while remaining numerically robust and computationally efficient. We derive an information-theoretic position uncertainty metric to quantify visual state-estimation quality and integrate it into the planner through three perception objectives: position uncertainty minimization, sequential field-of-view constraints, and look-ahead alignment. This enables systematic exploration of the trade-offs between speed and perceptual reliability. To accurately track the resulting perception-aware trajectories, we develop a model predictive contouring tracking controller that separates lateral and progress errors. Experiments demonstrate real-world flight speeds up to 9.8 m/s with 0.07 m average tracking error, and closed-loop success rates improved from 55% to 100% on a challenging Split-S course. The proposed system provides a scalable benchmark for studying the fundamental limits of perception-aware, time-optimal autonomous flight.

Figures (22)

• Figure 2: Our framework addresses the perception-aware, time-optimal planning problem for agile quadrotors. It supports both TOWP (blue) and TOGT (yellow) modes on a wide range of race tracks, and three perception objectives are proposed to ensure quality visual inputs. Tracks (a) and (b) demonstrate that our method can handle tracks defined in previous literature, while track (c) illustrates its superior flexibility in handling more complex scenarios where gates are in diverse shapes, sizes, and orientations.
• Figure 3: Illustrations of three perceptual objectives verified in the proposed framework. The arrows represent the camera optical directions, and the triangles with filled colors indicate the camera’s FOV. (a) In the absence of specific perception objectives, the camera's orientation is purely governed by time-optimal control, which could be detrimental to visual perception; (b) LA encourages the UAV to look at a future waypoint; (c) FOV keeps one or multiple target landmarks (e.g., the next racing gate) visible; (d) PUM trades off time optimality and information-theoretic position uncertainty for robust closed-loop execution.
• Figure 4: The framework accommodates gates representable by convex shapes, which constitute the majority of gates used in drone racing qin2023time.
• Figure 5: Diagram of quadrotor model with the world and body frames convention.
• Figure 6: Two types of geometry information for localization: (a) the relative geometry of multiple landmarks, where each landmark is treated as a single 3D point; or (b) features observable from a single landmark.

Theorems & Definitions (4)

• Proposition 1
• proof
• Proposition 2
• proof