Integrating spoken instructions into flight trajectory prediction to optimize automation in air traffic control

TL;DR

An automation paradigm integrating controlling intent into the information processing loop through the spoken instruction-aware flight trajectory prediction framework, which can formulate full-automated information processing in real-world air traffic applications, supporting human error detection and enhancing aviation safety.

Abstract

The booming air transportation industry inevitably burdens air traffic controllers' workload, causing unexpected human factor-related incidents. Current air traffic control systems fail to consider spoken instructions for traffic prediction, bringing significant challenges in detecting human errors during real-time traffic operations. Here, we present an automation paradigm integrating controlling intent into the information processing loop through the spoken instruction-aware flight trajectory prediction framework. A 3-stage progressive multi-modal learning paradigm is proposed to address the modality gap between the trajectory and spoken instructions, as well as minimize the data requirements. Experiments on a real-world dataset show the proposed framework achieves flight trajectory prediction with high predictability and timeliness, obtaining over 20% relative reduction in mean deviation error. Moreover, the generalizability of the proposed framework is also confirmed by various model architectures. The proposed framework can formulate full-automated information processing in real-world air traffic applications, supporting human error detection and enhancing aviation safety.

Figures (12)

• Figure 1: Comparison of conventional data-driven FTP and instruction-driven FTP. a An example of the ATC communication procedure and the challenges faced by the conventional FTP tools. b The logic flow of the proposed instruction-driven FTP paradigm.
• Figure 1: Error distribution of the FlightBERT++ and the proposed SIA-FTP. a The distribution of absolute percentage error for LLA attributes in 1, 3, 9, and 15 prediction horizons. b The distribution of square error for LLA attributes in 1, 3, 9, and 15 prediction horizons. Boxplots of a and b show the median (center line), and 1st and 3rd quartiles (Q1 and Q3, respectively). The error bars correspond to the Q1-(1.5*IQR) and Q3 + (1.5*IQR) range (IQR = Inter-Quartile Range). Data points below Q1 – (1.5*IQR) or above Q3 + (1.5*IQR) are considered outliers and not shown in the boxplots. Source data are provided as a Source Data file.
• Figure 2: Error distribution of the FlightBERT++ and the proposed SIA-FTP. a The distribution of absolute error for Longitude, Latitude, and Altitude attributes across different prediction horizons. b The distribution of deviation error across different prediction horizons. Boxplots of a and b show the median (center line), and 1st and 3rd quartiles (Q1 and Q3, respectively). The error bars correspond to the Q1-(1.5*IQR) and Q3 + (1.5*IQR) range (IQR = Inter-Quartile Range). Data points below Q1 – (1.5*IQR) or above Q3 + (1.5*IQR) are considered outliers and not shown in the boxplots. Source data are provided as a Source Data file.
• Figure 2: The MDE curve of the 15 prediction horizons for A1, A2 and SIA-FTP models. Source data are provided as a Source Data file.
• Figure 3: Visualization of the trajectory prediction results with different maneuvering controlling intents. a-b ALT_ADJ. c SPD_ADJ. d OFFSET. e CANOFF&FLYTO. f HEAD_ADJ&FLYTO. The ATC instructions are presented in the chatbox, where the color fonts indicate the keywords of the controlling intents. Source data are provided as a Source Data file.