Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CLiVR: Conversational Learning System in Virtual Reality with AI-Powered Patients

Akilan Amithasagaran, Sagnik Dakshit, Bhavani Suryadevara, Lindsey Stockton

TL;DR

The paper presents CLiVR, a VR-based clinical communication training system that leverages large language models, speech processing, and 3D avatars to simulate doctor–patient interactions. By grounding conversations in a curated syndrome–symptom knowledge base and providing real-time sentiment feedback, CLiVR aims to enhance realism, scalability, and reflective learning in medical education. An expert user study (n=13) demonstrates strong usability, perceived educational value, and willingness to integrate CLiVR while recognizing limitations in avatar realism and speech naturalness. The work positions CLiVR as a scalable complement to standardized patients, with clear avenues for expansion, such as richer clinical data integration and more diverse patient personas, to broaden adoption in clinical training programs.

Abstract

Simulations constitute a fundamental component of medical and nursing education and traditionally employ standardized patients (SP) and high-fidelity manikins to develop clinical reasoning and communication skills. However, these methods require substantial resources, limiting accessibility and scalability. In this study, we introduce CLiVR, a Conversational Learning system in Virtual Reality that integrates large language models (LLMs), speech processing, and 3D avatars to simulate realistic doctor-patient interactions. Developed in Unity and deployed on the Meta Quest 3 platform, CLiVR enables trainees to engage in natural dialogue with virtual patients. Each simulation is dynamically generated from a syndrome-symptom database and enhanced with sentiment analysis to provide feedback on communication tone. Through an expert user study involving medical school faculty (n=13), we assessed usability, realism, and perceived educational impact. Results demonstrated strong user acceptance, high confidence in educational potential, and valuable feedback for improvement. CLiVR offers a scalable, immersive supplement to SP-based training.

CLiVR: Conversational Learning System in Virtual Reality with AI-Powered Patients

TL;DR

The paper presents CLiVR, a VR-based clinical communication training system that leverages large language models, speech processing, and 3D avatars to simulate doctor–patient interactions. By grounding conversations in a curated syndrome–symptom knowledge base and providing real-time sentiment feedback, CLiVR aims to enhance realism, scalability, and reflective learning in medical education. An expert user study (n=13) demonstrates strong usability, perceived educational value, and willingness to integrate CLiVR while recognizing limitations in avatar realism and speech naturalness. The work positions CLiVR as a scalable complement to standardized patients, with clear avenues for expansion, such as richer clinical data integration and more diverse patient personas, to broaden adoption in clinical training programs.

Abstract

Simulations constitute a fundamental component of medical and nursing education and traditionally employ standardized patients (SP) and high-fidelity manikins to develop clinical reasoning and communication skills. However, these methods require substantial resources, limiting accessibility and scalability. In this study, we introduce CLiVR, a Conversational Learning system in Virtual Reality that integrates large language models (LLMs), speech processing, and 3D avatars to simulate realistic doctor-patient interactions. Developed in Unity and deployed on the Meta Quest 3 platform, CLiVR enables trainees to engage in natural dialogue with virtual patients. Each simulation is dynamically generated from a syndrome-symptom database and enhanced with sentiment analysis to provide feedback on communication tone. Through an expert user study involving medical school faculty (n=13), we assessed usability, realism, and perceived educational impact. Results demonstrated strong user acceptance, high confidence in educational potential, and valuable feedback for improvement. CLiVR offers a scalable, immersive supplement to SP-based training.
Paper Structure (21 sections, 2 figures, 3 tables)

This paper contains 21 sections, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. CLiVR System Overview
  4. Syndrome and Symptom Data Curation
  5. System Architectural Components
  6. LLM Roleplay and Prompt Engineering
  7. Sentiment Analysis of Medical Conversation
  8. Datasets
  9. Sentiment Evaluation
  10. Results and Discussion
  11. CLiVR Clinical Evaluation
  12. Study Design
  13. Study Questionnaire
  14. Study Results
  15. Qualitative Feedback Summary
  16. ...and 6 more sections

Figures (2)

  • Figure 1: End-to-end system architecture of the CLiVR application
  • Figure 2: Yes/No Response Summary (n = 13)