Combating Spatial Disorientation in a Dynamic Self-Stabilization Task Using AI Assistants

TL;DR

It is shown that certain AI assistants were able to improve human performance and that reinforcement-learning based assistants were objectively more effective but rated as less trusted and preferable by humans.

Abstract

Spatial disorientation is a leading cause of fatal aircraft accidents. This paper explores the potential of AI agents to aid pilots in maintaining balance and preventing unrecoverable losses of control by offering cues and corrective measures that ameliorate spatial disorientation. A multi-axis rotation system (MARS) was used to gather data from human subjects self-balancing in a spaceflight analog condition. We trained models over this data to create "digital twins" that exemplified performance characteristics of humans with different proficiency levels. We then trained various reinforcement learning and deep learning models to offer corrective cues if loss of control is predicted. Digital twins and assistant models then co-performed a virtual inverted pendulum (VIP) programmed with identical physics. From these simulations, we picked the 5 best-performing assistants based on task metrics such as crash frequency and mean distance from the direction of balance. These were used in a co-performance study with 20 new human subjects performing a version of the VIP task with degraded spatial information. We show that certain AI assistants were able to improve human performance and that reinforcement-learning based assistants were objectively more effective but rated as less trusted and preferable by humans.

Figures (5)

• Figure 1: Joystick deflections predicted by a DDPG (blue) and an LSTM trained over human data (green) compared to an actual 30-sec. MARS participant trial sample (participant deflections in red and angular position in black). This instance of the LSTM displays a test RMSE of .013 while the DDPG gets .803.
• Figure 2: Typical performance in the MARS and VIP tasks, before practice (Trial 1) and after practice (Trial 20). Phase plots show angular velocity vs. angular displacement from the DOB. The "standard" conditions provide angular displacement and velocity cues, and subjects improve significantly between first and last trial, seen as clustering around the origin (balance point) by Trial 20. The "disorienting" conditions eliminate sensory signals about displacement from the DOB, increasing positional drift (shown as phase loop oscillations around the X-axis) and destabilizing joystick commands that accelerate away from the DOB in the current direction of motion, with minimized learning and continued positional drift in Trial 20. Cyan dots indicate destabilizing deflections, where position, velocity and joystick deflection all have the same sign. Red dots denote anticipatory deflections, where position and joystick deflection have the same sign but velocity has the opposite sign---usually done to slow the IP down when velocity is perceived as being too high.
• Figure 3: Complementary evolution of discrete destabilizing and corrective commands as a function of angular deviation away from the DOB and toward a fall boundary, seen in MARS (red) and VIP (black) tasks.
• Figure 4: Model input and output structure. "Pilot/Assistant" stands in for any one of the trained prediction models.
• Figure 5: Absolute differences between baseline human performance metrics compared to AI-assistance in (a) Session 1 Task 2, (b) Session 2 Task 2 (different assistant model), and (c) Session 2 Task 3 (fine-tuned Session 2 Task 2 assistant).