Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments

TL;DR

This paper presents an immersive VR anatomy education environment augmented with a generative AI–driven embodied virtual assistant, comparing avatar-based and screen-based interactions across two cognitive complexity levels. In a within-subject pilot with 16 participants, the avatar configuration yielded higher scores for knowledge-based questions and elicited more interactions, while task completion time showed no robust differences. Usability, workload, and presence were similar between configurations, suggesting both modalities have value, and a hybrid approach could optimize learning. The study highlights the potential of embodied AI in education while noting limitations in analytical tasks and response accuracy, guiding future work toward larger samples, improved speech processing, and multimodal augmentation.

Abstract

Virtual reality (VR) and interactive 3D visualization systems have enhanced educational experiences and environments, particularly in complicated subjects such as anatomy education. VR-based systems surpass the potential limitations of traditional training approaches in facilitating interactive engagement among students. However, research on embodied virtual assistants that leverage generative artificial intelligence (AI) and verbal communication in the anatomy education context is underrepresented. In this work, we introduce a VR environment with a generative AI-embodied virtual assistant to support participants in responding to varying cognitive complexity anatomy questions and enable verbal communication. We assessed the technical efficacy and usability of the proposed environment in a pilot user study with 16 participants. We conducted a within-subject design for virtual assistant configuration (avatar- and screen-based), with two levels of cognitive complexity (knowledge- and analysis-based). The results reveal a significant difference in the scores obtained from knowledge- and analysis-based questions in relation to avatar configuration. Moreover, results provide insights into usability, cognitive task load, and the sense of presence in the proposed virtual assistant configurations. Our environment and results of the pilot study offer potential benefits and future research directions beyond medical education, using generative AI and embodied virtual agents as customized virtual conversational assistants.

Figures (5)

• Figure 1: System architecture of our proposed VR environment with the generative AI-based embodied virtual assistant.
• Figure 2: Overview of the study procedure. The order of the conditions (marked with $^*$) was counterbalanced.
• Figure 3: Results of user performance (n = 16), including (left) task completion time, (middle) number of interactions with the virtual assistant, and (right) score.
• Figure 4: Results of the questionnaires: (left) system usability scale (SUS) and (right) NASA task load index (TLX).
• Figure 5: Sense of presence results using Igroup presence questionnaire (IPQ) in the virtual environment.