Bayesian Data Augmentation and Training for Perception DNN in Autonomous Aerial Vehicles

TL;DR

This work addresses data-coverage gaps in perception for autonomous VTOLs by integrating a photorealistic simulation–driven data augmentation pipeline with Bayesian retraining of a YOLO-based landing detector. It combines reproducible dataset processing, uncertainty estimation, high-fidelity CARLA-GUAM simulation, and Gaussian Process–based Bayesian optimization to tune augmentation hyperparameters (e.g., scale and brightness) with the objective of maximizing landing success $f(oldsymbol{x})$. The approach yields a robust perception model that improves landing success from 50% to 70% across varied lighting and weather, and identifies a shared high-performing augmentation region that generalizes across conditions. This framework enhances VTOL safety and reliability in autonomous landing tasks and points toward real-world validation and broader environmental coverage as future work.

Abstract

Learning-based solutions have enabled incredible capabilities for autonomous systems. Autonomous vehicles, both aerial and ground, rely on DNN for various integral tasks, including perception. The efficacy of supervised learning solutions hinges on the quality of the training data. Discrepancies between training data and operating conditions result in faults that can lead to catastrophic incidents. However, collecting vast amounts of context-sensitive data, with broad coverage of possible operating environments, is prohibitively difficult. Synthetic data generation techniques for DNN allow for the easy exploration of diverse scenarios. However, synthetic data generation solutions for aerial vehicles are still lacking. This work presents a data augmentation framework for aerial vehicle's perception training, leveraging photorealistic simulation integrated with high-fidelity vehicle dynamics. Safe landing is a crucial challenge in the development of autonomous air taxis, therefore, landing maneuver is chosen as the focus of this work. With repeated simulations of landing in varying scenarios we assess the landing performance of the VTOL type UAV and gather valuable data. The landing performance is used as the objective function to optimize the DNN through retraining. Given the high computational cost of DNN retraining, we incorporated Bayesian Optimization in our framework that systematically explores the data augmentation parameter space to retrain the best-performing models. The framework allowed us to identify high-performing data augmentation parameters that are consistently effective across different landing scenarios. Utilizing the capabilities of this data augmentation framework, we obtained a robust perception model. The model consistently improved the perception-based landing success rate by at least 20% under different lighting and weather conditions.

Figures (10)

• Figure 1: Data Exploration Framework
• Figure 2: Initial Model Training
• Figure 3: Uncertainty Estimation in The System
• Figure 4: Object Detection, Tracking and Landing Controller
• Figure 5: Integration of perception, planning, and control with the simulator