Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI-Enhanced Virtual Reality in Medicine: A Comprehensive Survey

Yixuan Wu, Kaiyuan Hu, Danny Z. Chen, Jian Wu

TL;DR

This survey addresses the lack of a unified view of AI-enhanced VR in medicine by mapping methods across visualization, data analytics, and intervention. It introduces a three-fold taxonomy—Visualization Enhancement, VR-related Medical Data Processing, and VR-assisted Intervention—and details representative techniques within each category. Key topics include neural implicit reconstruction and diffusion models for 3D medical visualization, neural rendering and perception augmentation, multi-format 3D data analysis, and AI-guided intra-operative guidance and collaboration tools such as VQA/VQLA and scene graphs. The article also discusses data, ethical, and regulatory challenges and outlines future directions like NLP integration, biofeedback-enabled therapy, and real-time clinical analytics, positioning AI-VR as a foundational platform for future medical innovation.

Abstract

With the rapid advance of computer graphics and artificial intelligence technologies, the ways we interact with the world have undergone a transformative shift. Virtual Reality (VR) technology, aided by artificial intelligence (AI), has emerged as a dominant interaction media in multiple application areas, thanks to its advantage of providing users with immersive experiences. Among those applications, medicine is considered one of the most promising areas. In this paper, we present a comprehensive examination of the burgeoning field of AI-enhanced VR applications in medical care and services. By introducing a systematic taxonomy, we meticulously classify the pertinent techniques and applications into three well-defined categories based on different phases of medical diagnosis and treatment: Visualization Enhancement, VR-related Medical Data Processing, and VR-assisted Intervention. This categorization enables a structured exploration of the diverse roles that AI-powered VR plays in the medical domain, providing a framework for a more comprehensive understanding and evaluation of these technologies. To our best knowledge, this is the first systematic survey of AI-powered VR systems in medical settings, laying a foundation for future research in this interdisciplinary domain.

AI-Enhanced Virtual Reality in Medicine: A Comprehensive Survey

TL;DR

This survey addresses the lack of a unified view of AI-enhanced VR in medicine by mapping methods across visualization, data analytics, and intervention. It introduces a three-fold taxonomy—Visualization Enhancement, VR-related Medical Data Processing, and VR-assisted Intervention—and details representative techniques within each category. Key topics include neural implicit reconstruction and diffusion models for 3D medical visualization, neural rendering and perception augmentation, multi-format 3D data analysis, and AI-guided intra-operative guidance and collaboration tools such as VQA/VQLA and scene graphs. The article also discusses data, ethical, and regulatory challenges and outlines future directions like NLP integration, biofeedback-enabled therapy, and real-time clinical analytics, positioning AI-VR as a foundational platform for future medical innovation.

Abstract

With the rapid advance of computer graphics and artificial intelligence technologies, the ways we interact with the world have undergone a transformative shift. Virtual Reality (VR) technology, aided by artificial intelligence (AI), has emerged as a dominant interaction media in multiple application areas, thanks to its advantage of providing users with immersive experiences. Among those applications, medicine is considered one of the most promising areas. In this paper, we present a comprehensive examination of the burgeoning field of AI-enhanced VR applications in medical care and services. By introducing a systematic taxonomy, we meticulously classify the pertinent techniques and applications into three well-defined categories based on different phases of medical diagnosis and treatment: Visualization Enhancement, VR-related Medical Data Processing, and VR-assisted Intervention. This categorization enables a structured exploration of the diverse roles that AI-powered VR plays in the medical domain, providing a framework for a more comprehensive understanding and evaluation of these technologies. To our best knowledge, this is the first systematic survey of AI-powered VR systems in medical settings, laying a foundation for future research in this interdisciplinary domain.
Paper Structure (19 sections, 2 figures, 1 table)

This paper contains 19 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Taxonomy
  3. Visualization Enhancement
  4. VR-related Medical Data Processing
  5. VR-assisted Intervention
  6. Visualization Enhancement
  7. Virtual Reconstruction
  8. Visual Enhancement
  9. VR Medical Data Analytics
  10. Structural and Lesion Analysis
  11. Disease Diagnosis
  12. Surgical Support
  13. VR-assisted Intervention
  14. Intra-operative Intervention
  15. Interactive Collaboration
  16. ...and 4 more sections

Figures (2)

  • Figure 1: Demonstrating the workflow of AI-enhanced VR-assisted medical services. First, Visualization Enhancement focuses on the augmentation of a user’s visual and sensory perception within a virtual space. Next, VR-related Medical Data Processing discusses how VR, combined with AI, facilitates in-depth structural and lesion analysis, enhances disease diagnosis, and supports the entire stages of surgical procedures. Finally, VR-assisted Intervention covers the utilization of VR for direct procedural guidance, intra-operative intervention, and fostering interactive collaborations among medical professionals.
  • Figure 2: A taxonomy of AI-powered VR in medical services with representative examples.