Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

When Avatars Have Personality: Effects on Engagement and Communication in Immersive Medical Training

Julia S. Dollis, Iago A. Brito, Fernanda B. Färber, Pedro S. F. B. Ribeiro, Gustavo H. W. Barbosa, Andressa A. Bastos, Rafael T. Sousa, Arlindo R. Galvão Filho

TL;DR

This work tackles the lack of psychologically credible virtual patients in VR medical training by introducing a modular framework that embeds large language models to generate medically coherent virtual patients with distinct, consistent personalities. The system decouples personality from clinical data using four components (identity, backstory, personality profile, disease card) and implements it in a high-fidelity Unreal Engine VR environment. A mixed-methods within-subjects study with licensed physicians demonstrates feasibility, strong engagement, and nuanced insights such as the realism-verbosity paradox, while a large-scale synthetic evaluation (3,000 turns across 25 archetypes) validates disease and personality consistency. The findings underscore the value of personality diversity in training, provide design principles for believable agents, and offer a blueprint for scaling immersive, interpersonal medical training through LLM-driven digital humans.

Abstract

While virtual reality (VR) excels at simulating physical environments, its effectiveness for training complex interpersonal skills is limited by a lack of psychologically plausible virtual humans. This gap is particularly critical in medical education, where communication is a core clinical competency. This paper introduces a framework that integrates large language models (LLMs) into immersive VR to create medically coherent virtual patients with distinct, consistent personalities, based on a modular architecture that decouples personality from clinical data. We evaluated the system in a mixed-methods, within-subjects study with licensed physicians conducting simulated consultations. Results suggest that the approach is feasible and perceived as a rewarding and effective training enhancement. Our analysis highlights key design principles, including a "realism-verbosity paradox" and the importance of challenges being perceived as clinically authentic to support learning.

When Avatars Have Personality: Effects on Engagement and Communication in Immersive Medical Training

TL;DR

This work tackles the lack of psychologically credible virtual patients in VR medical training by introducing a modular framework that embeds large language models to generate medically coherent virtual patients with distinct, consistent personalities. The system decouples personality from clinical data using four components (identity, backstory, personality profile, disease card) and implements it in a high-fidelity Unreal Engine VR environment. A mixed-methods within-subjects study with licensed physicians demonstrates feasibility, strong engagement, and nuanced insights such as the realism-verbosity paradox, while a large-scale synthetic evaluation (3,000 turns across 25 archetypes) validates disease and personality consistency. The findings underscore the value of personality diversity in training, provide design principles for believable agents, and offer a blueprint for scaling immersive, interpersonal medical training through LLM-driven digital humans.

Abstract

While virtual reality (VR) excels at simulating physical environments, its effectiveness for training complex interpersonal skills is limited by a lack of psychologically plausible virtual humans. This gap is particularly critical in medical education, where communication is a core clinical competency. This paper introduces a framework that integrates large language models (LLMs) into immersive VR to create medically coherent virtual patients with distinct, consistent personalities, based on a modular architecture that decouples personality from clinical data. We evaluated the system in a mixed-methods, within-subjects study with licensed physicians conducting simulated consultations. Results suggest that the approach is feasible and perceived as a rewarding and effective training enhancement. Our analysis highlights key design principles, including a "realism-verbosity paradox" and the importance of challenges being perceived as clinically authentic to support learning.

Paper Structure

This paper contains 37 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Related Works
  3. Virtual Reality in Medical Training
  4. Personality Simulation in Digital Agents
  5. Methods
  6. VR Environment
  7. Virtual Patient Modeling
  8. Patient Identity
  9. Backstory
  10. Personality Profile
  11. Disease Card
  12. Patient Consistency
  13. Experiment Design
  14. Clinical Scenarios
  15. Participants
  16. ...and 22 more sections

Figures (1)

  • Figure 1: The architecture of our interactive patient simulation pipeline. The Virtual Patient Modeling stage (left) utilizes a real patient inquiry to generate a medically and behaviorally coherent patient. During the Physician Interaction stage (right), a patient response is generated using an LLM that is synthesized into speech via a Text-to-Speech (TTS) module. The corresponding Speech-to-Text (STT) module, which processes the physician's audio input, is omitted for visual clarity.