Transformer-Guided Deep Reinforcement Learning for Optimal Takeoff Trajectory Design of an eVTOL Drone
Nathan M. Roberts, Xiaosong Du
TL;DR
The paper addresses energy-efficient takeoff trajectory design for eVTOL aircraft under complex nonlinear dynamics. It introduces a transformer-guided deep reinforcement learning framework that uses temporal patterns learned from optimal trajectories to guide SAC-based policy learning, reducing exploration and accelerating convergence. Key results show the transformer-guided DRL achieving about 97% energy fidelity to a Dymos reference while requiring roughly a quarter of the training steps of vanilla DRL ($4.57\times10^6$ vs $19.79\times10^6$). This approach demonstrates data-efficient, high-quality trajectory optimization for eVTOL takeoff, with potential impact on practical deployment and certification of energy-efficient urban air mobility vehicles. The work lays groundwork for extending to broader flight conditions and safety constraints, and encourages exploration of other transformer/DRL design combinations for dynamic control problems.
Abstract
The rapid advancement of electric vertical take-off and landing (eVTOL) aircraft offers a promising opportunity to alleviate urban traffic congestion. Thus, developing optimal takeoff trajectories for minimum energy consumption becomes essential for broader eVTOL aircraft applications. Conventional optimal control methods (such as dynamic programming and linear quadratic regulator) provide highly efficient and well-established solutions but are limited by problem dimensionality and complexity. Deep reinforcement learning (DRL) emerges as a special type of artificial intelligence tackling complex, nonlinear systems; however, the training difficulty is a key bottleneck that limits DRL applications. To address these challenges, we propose the transformer-guided DRL to alleviate the training difficulty by exploring a realistic state space at each time step using a transformer. The proposed transformer-guided DRL was demonstrated on an optimal takeoff trajectory design of an eVTOL drone for minimal energy consumption while meeting takeoff conditions (i.e., minimum vertical displacement and minimum horizontal velocity) by varying control variables (i.e., power and wing angle to the vertical). Results presented that the transformer-guided DRL agent learned to take off with $4.57\times10^6$ time steps, representing 25% of the $19.79\times10^6$ time steps needed by a vanilla DRL agent. In addition, the transformer-guided DRL achieved 97.2% accuracy on the optimal energy consumption compared against the simulation-based optimal reference while the vanilla DRL achieved 96.3% accuracy. Therefore, the proposed transformer-guided DRL outperformed vanilla DRL in terms of both training efficiency as well as optimal design verification.