Deep Reinforcement Learning based Control Design for Aircraft Recovery from Loss-of-Control Scenario

TL;DR

This work addresses the challenge of recovering an aircraft from loss-of-control spins by formulating spin-recovery as a continuous-state, continuous-action Markov decision process and training a PPO agent in a high-fidelity 6DOF F18/HARV model with nonlinear aerodynamics and actuator saturation. A two-phase potential-based reward shaping guides learning from rapid angular-rate reduction to stable attitude alignment, achieving convergence after 6000 episodes and generalization to unseen upset initializations. The learned policy arrests $p$, $q$, and $r$, stabilizes $\alpha$, $\beta$, and $\mu$, and demonstrates competitive performance against a sliding-mode controller, highlighting the viability of deep RL for real-time, flight-critical recovery tasks. The study also outlines avenues for improvement, including recurrent architectures, safety constraints, and envelope-protection objectives to enhance generalization and robustness.

Abstract

Loss-of-control (LOC) remains a leading cause of fixed-wing aircraft accidents, especially in post-stall and flat-spin regimes where conventional gain-scheduled or logic-based recovery laws may fail. This study formulates spin-recovery as a continuous-state, continuous-action Markov Decision Process and trains a Proximal Policy Optimization (PPO) agent on a high-fidelity six-degree-of-freedom F-18/HARV model that includes nonlinear aerodynamics, actuator saturation and rate coupling. A two-phase potential-based reward structure first penalizes large angular rates and then enforces trimmed flight. After 6,000 simulated episodes, the policy generalities to unseen upset initializations. Results show that the learned policy successfully arrests the angular rates and stabilizes the angle of attack. The controller performance is observed to be satisfactory for recovery from spin condition which was compared with a state-of-the-art sliding mode controller. The findings demonstrate that deep reinforcement learning can deliver interpretable, dynamically feasible manoeuvres for real-time loss of control mitigation and provide a pathway for flight-critical RL deployment.

Figures (6)

• Figure 1: Proximal Policy Optimization Structure for Flight Control
• Figure 2: Training progress
• Figure 3: (a) Velocity of the Aircraft (ft/sec) (b) Angle of Attack (rads) (c) Angle of Sideslip (rads)
• Figure 4: (a) Roll rate (rad/sec) (b) Pitch rate (rad/sec) (c) Yaw rate (rad/sec)
• Figure 5: (a) Bank angle (rad) (b) Flight path angle (rad) (c) Altitude (ft)