From Novice to Skilled: RL-based Shared Autonomy Communicating with Pilots in UAV Multi-Task Missions

TL;DR

The paper tackles novice pilots' difficulty with depth perception in UAV multi-task missions (landing and inspection) by introducing a three-module shared autonomy system comprising perception (CM-SVAE latent embeddings), policy (Shared-TD3 with a Temporal-Encoder/Decoder), and information augmentation (LEDs and AR). It is trained entirely in simulation against simulated users and directly transferable to real pilots without retraining; the policy objective is to maximize the expected discounted return $\mathbb{E}\left[ \sum_t \gamma^t R_t \right]$. Key contributions include extending shared autonomy to multi-task UAV missions, learning pilot-system communication cues, and enabling efficient sim-to-real transfer through architectural innovations in TD3 and cross-domain perception. Empirical results from a physical user study ($n=29$) show task success rising from $16.67\%$ (landing) and $54.29\%$ (inspection) in the unassisted condition to $95.59\%$ and $96.49\%$ respectively with assistance, with red/green cues reducing inspection time by about $19.4\%$ and distance by about $17.8\%$; outdoor demonstrations in GPS-denied settings achieved $80\%$ and $85\%$ respectively, illustrating practical viability. The work demonstrates a viable path for deploying accessible UAV assistance that raises safety and efficiency for non-expert operators performing complex, sequential tasks.

Abstract

Multi-task missions for unmanned aerial vehicles (UAVs) involving inspection and landing tasks are challenging for novice pilots due to the difficulties associated with depth perception and the control interface. We propose a shared autonomy system, alongside supplementary information displays, to assist pilots to successfully complete multi-task missions without any pilot training. Our approach comprises of three modules: (1) a perception module that encodes visual information onto a latent representation, (2) a policy module that augments pilot's actions, and (3) an information augmentation module that provides additional information to the pilot. The policy module is trained in simulation with simulated users and transferred to the real world without modification in a user study (n=29), alongside alternative supplementary information schemes including learnt red/green light feedback cues and an augmented reality display. The pilot's intent is unknown to the policy module and is inferred from the pilot's input and UAV's states. The assistant increased task success rate for the landing and inspection tasks from [16.67% & 54.29%] respectively to [95.59% & 96.22%]. With the assistant, inexperienced pilots achieved similar performance to experienced pilots. Red/green light feedback cues reduced the required time by 19.53% and trajectory length by 17.86% for the inspection task, where participants rated it as their preferred condition due to the intuitive interface and providing reassurance. This work demonstrates that simple user models can train shared autonomy systems in simulation, and transfer to physical tasks to estimate user intent and provide effective assistance and information to the pilot.

Figures (13)

• Figure 1: System overview. (Top) Assistant network architecture. (Bottom left) Information augmentation using Microsoft HoloLens. (Bottom right) Example of a potential multi-task mission.
• Figure 1: Participant Demographics
• Figure 2: Architecture ablation study results. Each metric is averaged over the three training initialisations. For the top plots, a landing is considered a success if the UAV lands on the intended platform with all four legs contacting the surface with a relative yaw error of less than 20 degrees. For the bottom plots, an inspection is considered a success if all four corners of the intended platform are within the captured image with a relative yaw error of less than 20 degrees. Left column: model success rates whilst training. Middle column: model success rates during simulated validation. Right column: model success rates during physical validation.
• Figure 3: (Left) Example of the augmented reality display from the HoloLens. (Right) Example of red and green light feedback.
• Figure 4: Physical layout of user study environment. (Left) View of the arena from participant’s perspective. (Top right) UAV used in user study resting atop of landing platform. (Bottom right) Participants’ relative position to the arena.