Demonstrating Agile Flight from Pixels without State Estimation

TL;DR

This paper tackles the problem of autonomous agile quadrotor flight without explicit state estimation by training a vision-based policy that maps pixel inputs directly to low-level thrust and body-rate commands. The approach uses an asymmetric actor-critic with a privileged critic, a task-relevant inner-gate-edge abstraction for efficient pixel-based training, and a Swin-transformer gate detector for real-world deployment, enabling zero-shot sim-to-real transfer. Key contributions include the demonstrated capability to fly through racing gates at up to 40 km/h with 2 g accelerations, the robust training framework with domain randomization, and the deployment pipeline that relies on offboard computation while using standard hardware. The results show that the pixel-based policy closely approaches state-based performance in simulation and hardware-in-the-loop, and achieves 100% success in real-world trials, highlighting the practical potential of fully vision-based autonomous flight in structured environments and motivating broader applications beyond drone racing.

Abstract

Quadrotors are among the most agile flying robots. Despite recent advances in learning-based control and computer vision, autonomous drones still rely on explicit state estimation. On the other hand, human pilots only rely on a first-person-view video stream from the drone onboard camera to push the platform to its limits and fly robustly in unseen environments. To the best of our knowledge, we present the first vision-based quadrotor system that autonomously navigates through a sequence of gates at high speeds while directly mapping pixels to control commands. Like professional drone-racing pilots, our system does not use explicit state estimation and leverages the same control commands humans use (collective thrust and body rates). We demonstrate agile flight at speeds up to 40km/h with accelerations up to 2g. This is achieved by training vision-based policies with reinforcement learning (RL). The training is facilitated using an asymmetric actor-critic with access to privileged information. To overcome the computational complexity during image-based RL training, we use the inner edges of the gates as a sensor abstraction. This simple yet robust, task-relevant representation can be simulated during training without rendering images. During deployment, a Swin-transformer-based gate detector is used. Our approach enables autonomous agile flight with standard, off-the-shelf hardware. Although our demonstration focuses on drone racing, we believe that our method has an impact beyond drone racing and can serve as a foundation for future research into real-world applications in structured environments.

Figures (7)

• Figure 1: Our autonomous quadrotor can fly through a racetrack purely based on images from an onboard camera, without explicit state estimation. a) overview of our experimental setup consisting of racing gates, a quadrotor equipped with an onboard camera, and a video transmitter, b) onboard view from the camera, c) image abstraction used by the policy, d) overview over the racetrack.
• Figure 2: The architecture of our method consists of a gate detector, which is trained to segment the inner gate edges. The gate detection is downsampled to a size of 84$\times$84 and given as input to the three-layer CNN acting as a shared feature extractor for the asymmetric actor-critic framework. While training, we efficiently simulate the detected gates instead of using the detector. Both, the actor and critic are 2 hidden layer MLPs with 512 neurons each. The actor network has access to the current image encoding and the past three actions. The critic network, which is only used when training the policy, additionally receives privileged information about the state of the simulation environment.
• Figure 3: The gate detector is trained on data collected from real and synthetic environments; however, it has never seen real images from environment 2, in which the system is deployed (f).
• Figure 4: The top row shows the three different racetracks with trajectories flown by the pixel-based policy in the augmented simulator, while the bottom row shows the reward progress during training. With the asymmetric actor-critic architecture, the average reward of our pixel-based agent converges to a similar value as the state-based agent. Excluding the privileged information for the critic network results in unsuccessful learning. The training process requires roughly 3 hours for the state-based policy, while the pixel-based policies take approximately one day to train for 400 million steps.
• Figure 5: The rate of successful trials in the augmented simulator ablated by perturbing the gate position in positive and negative $x, y, z$ direction by a uniform distribution. While the state-based policy benefits from direct access to gate position information, the pixel-based policy demonstrates robustness by effectively inferring gate position solely from image observations, even in scenarios deviating from the training environment.