Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How LLMs are Shaping the Future of Virtual Reality

Süeda Özkaya, Santiago Berrezueta-Guzman, Stefan Wagner

TL;DR

The paper systematically surveys how Large Language Models reshape VR gaming by enabling emotionally intelligent NPCs, procedural storytelling, autonomous game masters, personalized player experiences, and accessible interfaces. It synthesizes 62 studies from 2018–2025, mapping applications to six domains and detailing technical and ethical challenges such as latency, memory, bias, privacy, and immersion. The analysis highlights that while LLMs enhance realism and engagement, real‑time performance, memory management, and responsible deployment remain critical barriers, motivating hybrid architectures, multimodal integration, and ethical safeguards. The work outlines future directions in multimodal AI, reinforcement learning, affective computing, and open‑source collaboration, aiming to guide scalable, inclusive, and trustworthy intelligent VR systems with practical impact for developers and researchers.

Abstract

The integration of Large Language Models (LLMs) into Virtual Reality (VR) games marks a paradigm shift in the design of immersive, adaptive, and intelligent digital experiences. This paper presents a comprehensive review of recent research at the intersection of LLMs and VR, examining how these models are transforming narrative generation, non-player character (NPC) interactions, accessibility, personalization, and game mastering. Drawing from an analysis of 62 peer reviewed studies published between 2018 and 2025, we identify key application domains ranging from emotionally intelligent NPCs and procedurally generated storytelling to AI-driven adaptive systems and inclusive gameplay interfaces. We also address the major challenges facing this convergence, including real-time performance constraints, memory limitations, ethical risks, and scalability barriers. Our findings highlight that while LLMs significantly enhance realism, creativity, and user engagement in VR environments, their effective deployment requires robust design strategies that integrate multimodal interaction, hybrid AI architectures, and ethical safeguards. The paper concludes by outlining future research directions in multimodal AI, affective computing, reinforcement learning, and open-source development, aiming to guide the responsible advancement of intelligent and inclusive VR systems.

How LLMs are Shaping the Future of Virtual Reality

TL;DR

The paper systematically surveys how Large Language Models reshape VR gaming by enabling emotionally intelligent NPCs, procedural storytelling, autonomous game masters, personalized player experiences, and accessible interfaces. It synthesizes 62 studies from 2018–2025, mapping applications to six domains and detailing technical and ethical challenges such as latency, memory, bias, privacy, and immersion. The analysis highlights that while LLMs enhance realism and engagement, real‑time performance, memory management, and responsible deployment remain critical barriers, motivating hybrid architectures, multimodal integration, and ethical safeguards. The work outlines future directions in multimodal AI, reinforcement learning, affective computing, and open‑source collaboration, aiming to guide scalable, inclusive, and trustworthy intelligent VR systems with practical impact for developers and researchers.

Abstract

The integration of Large Language Models (LLMs) into Virtual Reality (VR) games marks a paradigm shift in the design of immersive, adaptive, and intelligent digital experiences. This paper presents a comprehensive review of recent research at the intersection of LLMs and VR, examining how these models are transforming narrative generation, non-player character (NPC) interactions, accessibility, personalization, and game mastering. Drawing from an analysis of 62 peer reviewed studies published between 2018 and 2025, we identify key application domains ranging from emotionally intelligent NPCs and procedurally generated storytelling to AI-driven adaptive systems and inclusive gameplay interfaces. We also address the major challenges facing this convergence, including real-time performance constraints, memory limitations, ethical risks, and scalability barriers. Our findings highlight that while LLMs significantly enhance realism, creativity, and user engagement in VR environments, their effective deployment requires robust design strategies that integrate multimodal interaction, hybrid AI architectures, and ethical safeguards. The paper concludes by outlining future research directions in multimodal AI, affective computing, reinforcement learning, and open-source development, aiming to guide the responsible advancement of intelligent and inclusive VR systems.

Paper Structure

This paper contains 24 sections, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Large Language Models (LLMs)--Overview
  4. Virtual Reality Games--Overview
  5. The Intersection of LLMs and VR Games
  6. Methodology
  7. Literature Search Strategy
  8. Key Applications of LLMs in VR Games
  9. Dynamic NPC Interactions and Emotional Intelligence
  10. Procedural Storytelling and Narrative Generation
  11. Intelligent Game Masters for VR and Adaptive Systems
  12. Personalized Player Experience
  13. Accessibility, Inclusivity, and Usability
  14. Challenges and Limitations
  15. Computational and Performance Constraints
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Overview of the paper structure and organization.
  • Figure 2: Application areas of Large Language Models in Virtual Reality games
  • Figure 3: Distribution of reviewed papers by publication year.
  • Figure 4: Categorization of reviewed papers based on key application areas. Note that one paper may be relevant to multiple categories.