SafeSpect: Safety-First Augmented Reality Heads-up Display for Drone Inspections

TL;DR

This work addresses the cognitive load and situational-awareness challenges faced by drone pilots during façade inspections by proposing a safety-first adaptive AR heads-up display. Through participatory design with professional pilots and a VR drone-inspection simulator, the authors develop an adaptive AR interface that switches between mission- and safety-focused views, and compare it against 2D and non-adaptive AR baselines. Results show that the adaptive AR reduces cognitive load and enhances situational awareness relative to a 2D baseline, with stronger safety information saliency and user confidence when AR is fully leveraged; however, full AR without adaptation can overwhelm users and adaptive behavior requires trust. The study provides design guidelines and emphasizes transparency of adaptive rules, saliency of warnings, and user customization to improve safety-critical AR guidance for drone operations, with potential applicability to other safety-sensitive robotics tasks.

Abstract

Current tablet-based interfaces for drone operations often impose a heavy cognitive load on pilots and reduce situational awareness by dividing attention between the video feed and the real world. To address these challenges, we designed a heads-up augmented reality (AR) interface that overlays in-situ information to support drone pilots in safety-critical tasks. Through participatory design workshops with professional pilots, we identified key features and developed an adaptive AR interface that dynamically switches between task and safety views to prevent information overload. We evaluated our prototype by creating a realistic building inspection task and comparing three interfaces: a 2D tablet, a static AR, and our adaptive AR design. A user study with 15 participants showed that the AR interface improved access to safety information, while the adaptive AR interface reduced cognitive load and enhanced situational awareness without compromising task performance. We offer design insights for developing safety-first heads-up AR interfaces.

Figures (13)

• Figure 1: Examples of the safety and mission-related visualization illustrated by the pilots. (a) Display the heading of the drone, illustrated by D3. (b) Show the nearby waypoints and the photo-taking points over the trajectory, illustrated by D1. (c) Allow marking the point of interest from the camera view while monitoring, illustrated by D3. (d) Visualize safety signals at the peripheral vision, illustrated by D2. (e) Show the RTH path with battery remaining energy, illustrated by D1. (f) Display a boundary around the drone and the ground projection, illustrated by D4.
• Figure 2: The simulated building inspection scenario and different types of defects used in our study. (a) Overall configuration of the scenario. (b) Paint peel. (c) Wall crack. (d) Wall dent. (e) Rotten surface. (f) Leakage. (g) Delamination.
• Figure 3: Interactive 2D Camera View. Left: Inputs from the Meta Quest Touch Pro VR Controllers (Only the Left Controller is shown in the picture). Right: Interactive buttons on the interface. The autopilot button (bottom right) will be highlighted if the autopilot is on.
• Figure 4: Mid-air obstacles. (a) A tall tree. (b) A gondola machine.
• Figure 5: (a) Mission boundary (in green). (b) Generated path plan (in blue) based on building geometry and parameters. (c) Waypoints and coverage visualization during autpilot.