Continual Learning for Robust Gate Detection under Dynamic Lighting in Autonomous Drone Racing

TL;DR

The paper tackles robust gate detection for autonomous drone racing under dynamic lighting by introducing a continual-learning–enabled perception pipeline built on a lightweight backbone. It formulates gate pose estimation from monocular imagery as predicting image-space center, distance, and yaw to reconstruct 3D gate poses, using pencil-filtered inputs and a grid-based output. Through a dataset of 6,760 samples across five illumination levels and multiple CL strategies (PredKD, FeatKD, ER, DER, DER++), the study demonstrates that rehearsal-based methods, particularly DER++, effectively mitigate catastrophic forgetting and improve robustness across lighting conditions. The findings highlight the importance of task sequencing and suggest online continual learning as a fruitful direction for real-time adaptation in dynamic environments.

Abstract

In autonomous and mobile robotics, a principal challenge is resilient real-time environmental perception, particularly in situations characterized by unknown and dynamic elements, as exemplified in the context of autonomous drone racing. This study introduces a perception technique for detecting drone racing gates under illumination variations, which is common during high-speed drone flights. The proposed technique relies upon a lightweight neural network backbone augmented with capabilities for continual learning. The envisaged approach amalgamates predictions of the gates' positional coordinates, distance, and orientation, encapsulating them into a cohesive pose tuple. A comprehensive number of tests serve to underscore the efficacy of this approach in confronting diverse and challenging scenarios, specifically those involving variable lighting conditions. The proposed methodology exhibits notable robustness in the face of illumination variations, thereby substantiating its effectiveness.

Figures (6)

• Figure 1: Illustration of the operational principles of the gate perception framework. Images of the racing track are captured using a single fish-eye camera under various illumination conditions. The proposed framework incorporates continual learning methodologies, whereby the neural network undergoes incremental training to detect the gates in various conditions without forgetting any previous conditions.
• Figure 2: Illustration of the gate perception pipeline. A raw RGB image is converted by the pencil filter before inputting into the neural network, which predicts the center of the gate in the image frame, along with the corresponding distance and orientation of the gate relative to the drone's body frame. Using this information, a 3D pose of the gate can be reconstructed via backprojection.
• Figure 3: Samples from datasets with environmental illumination varying from $100\%$ to $10\%$ of light intensity.
• Figure 4: Samples from datasets. Top row: RGB images with illumination varying from $10\%$ to $100\%$. Bottom row: images after applying pencil filter.
• Figure 5: Evolution of Average Error for bright-to-dark training sequence.