Integrating Personality into Digital Humans: A Review of LLM-Driven Approaches for Virtual Reality

TL;DR

The paper addresses how to embed stable, human-like personality into VR digital humans using LLM-based generation and multimodal signals. It reviews methods for modelling personality via zero-shot, few-shot, and fine-tuning, and surveys qualitative and quantitative evaluation approaches while highlighting gaps in multimodal VR contexts. Key findings point to substantial computational and latency barriers, as well as a lack of standardized evaluation frameworks for embodied agents. The work underscores the potential for education, therapy, and gaming applications and calls for interdisciplinary collaboration to advance human-computer interaction in immersive environments.

Abstract

The integration of large language models (LLMs) into virtual reality (VR) environments has opened new pathways for creating more immersive and interactive digital humans. By leveraging the generative capabilities of LLMs alongside multimodal outputs such as facial expressions and gestures, virtual agents can simulate human-like personalities and emotions, fostering richer and more engaging user experiences. This paper provides a comprehensive review of methods for enabling digital humans to adopt nuanced personality traits, exploring approaches such as zero-shot, few-shot, and fine-tuning. Additionally, it highlights the challenges of integrating LLM-driven personality traits into VR, including computational demands, latency issues, and the lack of standardized evaluation frameworks for multimodal interactions. By addressing these gaps, this work lays a foundation for advancing applications in education, therapy, and gaming, while fostering interdisciplinary collaboration to redefine human-computer interaction in VR.