Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Toward Scalable Patient Safety Training: A Prototype for Root Cause Analysis Simulation With AI Virtual Avatars

Yuqi Hu, Qiwen Xiong, Zhenzhen Qin, Brandon Watanabe, Yujing Wang, Mirjana Prpa, Ilmi Yoon

Abstract

Patient safety training is essential for preparing healthcare professionals to identify, investigate, and prevent adverse events. However, conventional simulation-based approaches often require substantial faculty time, physical resources, and standardized facilitation. This paper presents a prototype AI-powered simulation platform designed to support more scalable patient safety training through root cause analysis (RCA). The system provides a Unity-based 3D simulation environment, which allows trainees to investigate an ICU adverse event by interviewing five virtual team members represented as AI-powered avatars. Each avatar is driven by a large language model (LLM) agent with role-specific knowledge and variable states of mind. Moreover, emotional text-to-speech and AI-supported facial and body animation enable more realistic and immersive interactions. After completing the simulation, trainees submit a written RCA report and receive rubric-guided formative and summative feedback automatically generated by an LLM-based assessment component. The prototype is built to support patient safety training for healthcare professionals, focusing on skills in communication, investigation, thinking, and analysis, with low recurring instructional burden. We describe the design of the platform, its core technical components, and an RCA case based on a published ICU scenario. This work demonstrates the feasibility of integrating generative AI into immersive simulation for scalable patient safety education.

Toward Scalable Patient Safety Training: A Prototype for Root Cause Analysis Simulation With AI Virtual Avatars

Abstract

Patient safety training is essential for preparing healthcare professionals to identify, investigate, and prevent adverse events. However, conventional simulation-based approaches often require substantial faculty time, physical resources, and standardized facilitation. This paper presents a prototype AI-powered simulation platform designed to support more scalable patient safety training through root cause analysis (RCA). The system provides a Unity-based 3D simulation environment, which allows trainees to investigate an ICU adverse event by interviewing five virtual team members represented as AI-powered avatars. Each avatar is driven by a large language model (LLM) agent with role-specific knowledge and variable states of mind. Moreover, emotional text-to-speech and AI-supported facial and body animation enable more realistic and immersive interactions. After completing the simulation, trainees submit a written RCA report and receive rubric-guided formative and summative feedback automatically generated by an LLM-based assessment component. The prototype is built to support patient safety training for healthcare professionals, focusing on skills in communication, investigation, thinking, and analysis, with low recurring instructional burden. We describe the design of the platform, its core technical components, and an RCA case based on a published ICU scenario. This work demonstrates the feasibility of integrating generative AI into immersive simulation for scalable patient safety education.

Paper Structure

This paper contains 13 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Simulation-Based Patient Safety Training for Healthcare Professionals
  4. LLMs in Healthcare Training and Education
  5. Prototype Design and Implementation
  6. Simulation Scenario: ICU Failure Case Study
  7. LLM-Based Virtual Avatars
  8. Emotional Text-to-Speech
  9. AI-Powered Character Animations
  10. LLM-Based Assessment of Student Performance
  11. Discussion
  12. Limitations
  13. Future Work

Figures (3)

  • Figure 1: System architecture of the Unity-based virtual RCA training platform and interactions between components during simulation.
  • Figure 2: Prototype illustrations. Trainees can view background on the case and instructions for conducting the RCA before staring. (top-left). Interviews are conducted via speech-to-speech conversations with virtual avatars, who will have different facial and body animations depending on their emotion state. For example, they will have more dramatic gestures when irritated (top-right) than calm (bottom-left). The session concludes with an LLM-generated assessment report based on the trainee’s RCA submission (bottom-right).
  • Figure 3: The process of generating customized animations. Left: motions performed by real humans. Middle: skeletal animations generated by Move.ai. Right: animations applied to 3D avatars in Unity.