Adversarial Reinforcement Learning for Robust Control of Fixed-Wing Aircraft under Model Uncertainty

TL;DR

This work addresses robust path-following for a fixed-wing sUAS under aerodynamic model uncertainty by training a controller with Robust Adversarial Reinforcement Learning (RARL). An adversary applies rate-bounded perturbations to aerodynamic coefficients to create challenging scenarios, and the protagonist is trained via Proximal Policy Optimization (PPO) to achieve accurate, robust path-tracking. Compared to controllers trained with stochastic disturbances and to a switched uncertain-initial-condition baseline, the adversarially trained controller delivers tighter tracking, lower control effort, and reduced worst-case deviations in high-fidelity 6DOF CZ-150 simulations. The results demonstrate that adversarial training enhances generalization to unforeseen disturbances and offers tangible robustness improvements for real-world sUAS trajectory-tracking, with future work focusing on broader uncertainties and flight testing.

Abstract

This paper presents a reinforcement learning-based path-following controller for a fixed-wing small uncrewed aircraft system (sUAS) that is robust to uncertainties in the aerodynamic model of the sUAS. The controller is trained using the Robust Adversarial Reinforcement Learning framework, where an adversary perturbs the environment (aerodynamic model) to expose the agent (sUAS) to demanding scenarios. In our formulation, the adversary introduces rate-bounded perturbations to the aerodynamic model coefficients. We demonstrate that adversarial training improves robustness compared to controllers trained using stochastic model uncertainty. The learned controller is also benchmarked against a switched uncertain initial condition controller. The effectiveness of the approach is validated through high-fidelity simulations using a realistic six-degree-of-freedom fixed-wing aircraft model, showing accurate and robust path-following performance under a variety of uncertain aerodynamic conditions.

Figures (4)

• Figure 1: Aggregate of performance metrics for controllers trained under stochastic and adversarial aerodynamic uncertainties.
• Figure 2: Switched UIC and RL-based controllers evaluated against stochastic and adversarial aerodynamic uncertainties.
• Figure 3: Adversarial aerodynamic model perturbations for an example simulation episode.
• Figure 4: State and control input histories of the switched UIC controller and RL controller for an example simulation episode.