Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LeRAAT: LLM-Enabled Real-Time Aviation Advisory Tool

Marc R. Schlichting, Vale Rasmussen, Heba Alazzeh, Houjun Liu, Kiana Jafari, Amelia F. Hardy, Dylan M. Asmar, Mykel J. Kochenderfer

TL;DR

LeRAAT tackles the challenge of providing real-time, context-specific pilot advisory in emergencies by fusing live flight data, weather, and aircraft documentation through a Retrieval-Augmented Generation pipeline within an X-Plane simulation. The system architecture comprises a modular UI plugin, a relay server, and an LLM backend (GPT-4o) with a RAG framework that sources aircraft manuals and regulatory materials, grounded by a FAISS vector store. Pilot feedback from SME evaluations in Boeing-style two scenarios indicates the approach can improve situational awareness and accelerate decision-making, particularly in diversion assessments, while highlighting the need for voice interfaces and richer data sources like NOTAMs. The work demonstrates a concrete path toward AI-assisted cockpit decision support and identifies practical next steps for broader validation and safety enhancements in aviation contexts.

Abstract

In aviation emergencies, high-stakes decisions must be made in an instant. Pilots rely on quick access to precise, context-specific information -- an area where emerging tools like large language models (LLMs) show promise in providing critical support. This paper introduces LeRAAT, a framework that integrates LLMs with the X-Plane flight simulator to deliver real-time, context-aware pilot assistance. The system uses live flight data, weather conditions, and aircraft documentation to generate recommendations aligned with aviation best practices and tailored to the particular situation. It employs a Retrieval-Augmented Generation (RAG) pipeline that extracts and synthesizes information from aircraft type-specific manuals, including performance specifications and emergency procedures, as well as aviation regulatory materials, such as FAA directives and standard operating procedures. We showcase the framework in both a virtual reality and traditional on-screen simulation, supporting a wide range of research applications such as pilot training, human factors research, and operational decision support.

LeRAAT: LLM-Enabled Real-Time Aviation Advisory Tool

TL;DR

LeRAAT tackles the challenge of providing real-time, context-specific pilot advisory in emergencies by fusing live flight data, weather, and aircraft documentation through a Retrieval-Augmented Generation pipeline within an X-Plane simulation. The system architecture comprises a modular UI plugin, a relay server, and an LLM backend (GPT-4o) with a RAG framework that sources aircraft manuals and regulatory materials, grounded by a FAISS vector store. Pilot feedback from SME evaluations in Boeing-style two scenarios indicates the approach can improve situational awareness and accelerate decision-making, particularly in diversion assessments, while highlighting the need for voice interfaces and richer data sources like NOTAMs. The work demonstrates a concrete path toward AI-assisted cockpit decision support and identifies practical next steps for broader validation and safety enhancements in aviation contexts.

Abstract

In aviation emergencies, high-stakes decisions must be made in an instant. Pilots rely on quick access to precise, context-specific information -- an area where emerging tools like large language models (LLMs) show promise in providing critical support. This paper introduces LeRAAT, a framework that integrates LLMs with the X-Plane flight simulator to deliver real-time, context-aware pilot assistance. The system uses live flight data, weather conditions, and aircraft documentation to generate recommendations aligned with aviation best practices and tailored to the particular situation. It employs a Retrieval-Augmented Generation (RAG) pipeline that extracts and synthesizes information from aircraft type-specific manuals, including performance specifications and emergency procedures, as well as aviation regulatory materials, such as FAA directives and standard operating procedures. We showcase the framework in both a virtual reality and traditional on-screen simulation, supporting a wide range of research applications such as pilot training, human factors research, and operational decision support.

Paper Structure

This paper contains 10 sections, 3 figures.

Table of Contents

  1. Introduction
  2. System Architecture
  3. X-Plane Integration.
  4. Interactive UI Components.
  5. Relay Server.
  6. LLM Backend.
  7. Data Flow and Processing.
  8. Retrieval-Augmented Generation.
  9. Example Scenarios and Pilot Feedback
  10. Conclusion and Future Work

Figures (3)

  • Figure 1: LeRAAT system architecture. Highlighted in blue are our main contributions.
  • Figure 2: Graphical user interface with LLM response to a full authority digital engine control (FADEC) failure.
  • Figure 3: Discrete state model of LeRAAT.