Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Demonstrating PilotAR: A Tool to Assist Wizard-of-Oz Pilot Studies with OHMD

Nuwan Janaka, Runze Cai, Shengdong Zhao, David Hsu

TL;DR

PilotAR tackles the difficulty of conducting rapid, in-context AR/MR pilot studies with Optical See-Through Head-Mounted Displays by delivering an open-source desktop solution that unifies live FPV/TPV observation, multi-modal annotations, and end-to-end study workflow. The tool supports real-time data summaries, multi-experimenter coordination, and post-hoc analysis/export, enabling faster concept validation and collaborative interpretation. Its three-phase workflow (pre-pilot, during-pilot, post-pilot) and flexible wizarding interfaces reduce development time and facilitate iterative WOz experiments with OHMDs. By aggregating observation, annotation, and analysis in one platform, PilotAR enhances data quality, discussion depth, and community sharing, with implications for accelerating AR/MR research and beyond-pilot applications.

Abstract

While pilot studies help to identify potential interesting research directions, the additional requirements in AR/MR make it challenging to conduct quick and dirty pilot studies efficiently with Optical See-Through Head-Mounted Displays (OST HMDs, OHMDs). To overcome these challenges, including the inability to observe and record in-context user interactions, increased task load, and difficulties with in-context data analysis and discussion, we introduce PilotAR (https://github.com/Synteraction-Lab/PilotAR), a tool designed iteratively to enhance AR/MR pilot studies, allowing live first-person and third-person views, multi-modal annotations, flexible wizarding interfaces, and multi-experimenter support.

Demonstrating PilotAR: A Tool to Assist Wizard-of-Oz Pilot Studies with OHMD

TL;DR

PilotAR tackles the difficulty of conducting rapid, in-context AR/MR pilot studies with Optical See-Through Head-Mounted Displays by delivering an open-source desktop solution that unifies live FPV/TPV observation, multi-modal annotations, and end-to-end study workflow. The tool supports real-time data summaries, multi-experimenter coordination, and post-hoc analysis/export, enabling faster concept validation and collaborative interpretation. Its three-phase workflow (pre-pilot, during-pilot, post-pilot) and flexible wizarding interfaces reduce development time and facilitate iterative WOz experiments with OHMDs. By aggregating observation, annotation, and analysis in one platform, PilotAR enhances data quality, discussion depth, and community sharing, with implications for accelerating AR/MR research and beyond-pilot applications.

Abstract

While pilot studies help to identify potential interesting research directions, the additional requirements in AR/MR make it challenging to conduct quick and dirty pilot studies efficiently with Optical See-Through Head-Mounted Displays (OST HMDs, OHMDs). To overcome these challenges, including the inability to observe and record in-context user interactions, increased task load, and difficulties with in-context data analysis and discussion, we introduce PilotAR (https://github.com/Synteraction-Lab/PilotAR), a tool designed iteratively to enhance AR/MR pilot studies, allowing live first-person and third-person views, multi-modal annotations, flexible wizarding interfaces, and multi-experimenter support.
Paper Structure (16 sections, 4 figures, 1 table)

This paper contains 16 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction and Related Work
  2. PilotAR Tool
  3. Major Functions
  4. PilotAR Usage Scenario
  5. Interface and Workflow
  6. Pre-pilot Phase
  7. During-pilot Phase
  8. Post-pilot Phase
  9. Conclusion
  10. Comparison
  11. Implementation
  12. PilotAR Setup
  13. Role Selection (Figure \ref{['fig:tool:setup_ui']}A)
  14. Device Configuration (Figure \ref{['fig:tool:setup_ui']}C1)
  15. Checklist Creation (Figure \ref{['fig:tool:setup_ui']}C2)
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Overview of the system components and workflow with PilotAR.
  • Figure 2: Pilot interface, which includes two major areas. Area (A) is the Top Bar showing (pinned) Annotations' live statistics (A1), the session progress (A2), and session information (A3). Area (B) presents the main working panel housing the FPV (B1, which shows the digital interface and user's feet from their FPV), TPV (B2), Wizarding Interface (B3), and a sidebar for the annotation table (B4).
  • Figure 3: The Analyzer interface comprises two main panels: the video panel (A) and the annotation panel (B). The video panel includes video playbacks of the pilot (A1), video controls, and a new note panel (A2). The annotation panel features an annotation preview (B1), annotation filtering options (B2), an annotation table (B3), and an exporting button. The Analyzer supports exporting the annotations (C) in PDF format (C1) and CSV format (C2).
  • Figure 4: Workflow of Setup UI. Upon starting the tool, the experimenter is prompted to select the role (A), including single- and multi-experimenter (wizard/observer). Then, menu (B) indicates the three major steps of conducting a pilot study: Setup, Pilot, and Analyzer. In Setup (C), there are three sub-steps, including device configurations (C1), checklist configuration (C2), and annotation customization (C3).