Co-Designing Augmented Reality Tools for High-Stakes Clinical Teamwork

TL;DR

The paper tackles the challenge of improving high-stakes ED teamwork through augmented reality head-mounted displays (AR-HMDs) by conducting a 11-month, IRB-approved, participatory co-design study with 10 healthcare workers. It uses a two-phase process (challenge elicitation and iterative ideation with storyboards) and thematic analysis to derive seven role-based AR-HMD application scenarios and practical design guidelines. The work highlights how AR-HMDs can support information sharing, latecomer information retrieval, and remote guidance while identifying barriers such as cognitive overload, unfamiliarity, and workflow integration. The findings offer actionable design insights to develop AR-HMDs that enhance team collaboration and reduce errors in acute care, and they outline a path for future work including VR prototyping, simulation-based evaluation, and micro-interaction approaches.

Abstract

How might healthcare workers (HCWs) leverage augmented reality head-mounted displays (AR-HMDs) to enhance teamwork? Although AR-HMDs have shown immense promise in supporting teamwork in healthcare settings, design for Emergency Department (ER) teams has received little attention. The ER presents unique challenges, including procedural recall, medical errors, and communication gaps. To address this gap, we engaged in a participatory design study with healthcare workers to gain a deep understanding of the potential for AR-HMDs to facilitate teamwork during ER procedures. Our results reveal that AR-HMDs can be used as an information-sharing and information-retrieval system to bridge knowledge gaps, and concerns about integrating AR-HMDs in ER workflows. We contribute design recommendations for seven role-based AR-HMD application scenarios involving HCWs with various expertise, working across multiple medical tasks. We hope our research inspires designers to embark on the development of new AR-HMD applications for high-stakes, team environments.

Figures (5)

• Figure 1: This shows the Zoom-based study workflow including two phases: Phase I reflected on teamwork challenges and ways augmented reality devices can help and Phase II focused on envisioning new AR-HMD interactions and reflecting on ways for it to support team collaboration.
• Figure 2: Example storyboards generated in Phase I. This shows an example of using AR-HMDs to guide healthcare workers (HCWs) through medical procedures, starting with handing off patient information from Emergency Medical Services to HCWs in the Emergency Room and using AR-HMDs to visualize tasks for HCWs and walk them through steps to treat a patient based on standard procedural protocols including timings to start and end a sub-tasks in a medical procedure, identify medical equipment needed for sub-tasks, and potential deviations from standard procedure based on pre-existing conditions.
• Figure 3: Design sketches for procedural guidance. a) AR-HMDs that show CPR rounds (figure instructs HCW to perform round 3 of total 4) and time until it must be completed (3 minutes and 26 seconds). b) AR-HMDs for administering medications, displaying patient vital signs, and task bundles.
• Figure 4: AR-HMDs for rare procedures. a) Similar to the Lucas device that is often positioned on the crash cart, the AR-HMDs can also be placed in this location. AR-HMDs can be used to confirm patient consent, communicate with the crash cart to help locate medications or other tools in the cart, and walk users through rare procedures, including administering medications not often provided to patients. b) AR-HMDs for the team leader to track patient vital signs and pending tasks (left) and an individual interface for the team members to track tasks specifically assigned to them.
• Figure 5: Role-specific AR-HMD applications for acute care teams.