Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Visualization of AI Systems in Virtual Reality: A Comprehensive Review

Medet Inkarbekov, Rosemary Monahan, Barak A. Pearlmutter

TL;DR

This paper surveys the visualization of AI systems in virtual reality by reviewing 18 peer‑reviewed studies up to January 2023. It analyzes how VR engines (notably Unity and Unreal) are used alongside AI backends (TensorFlow, PyTorch, etc.) to render and interact with neural networks and related models, while identifying challenges such as data dimensionality, cognitive load, and motion sickness. The authors synthesize visualization and interaction techniques (e.g., 3D network visualizations, feature-map views, immersive node-link diagrams) and discuss domain applications, implementation hurdles, and ethical considerations. The study provides a roadmap for future research, emphasizing more intuitive tools, standardization, cross‑tool comparability, and responsible integration of VR and AI technologies to unlock broader adoption and impact.

Abstract

This study provides a comprehensive review of the utilization of Virtual Reality (VR) for visualizing Artificial Intelligence (AI) systems, drawing on 18 selected studies. The results illuminate a complex interplay of tools, methods, and approaches, notably the prominence of VR engines like Unreal Engine and Unity. However, despite these tools, a universal solution for effective AI visualization remains elusive, reflecting the unique strengths and limitations of each technique. We observed the application of VR for AI visualization across multiple domains, despite challenges such as high data complexity and cognitive load. Moreover, it briefly discusses the emerging ethical considerations pertaining to the broad integration of these technologies. Despite these challenges, the field shows significant potential, emphasizing the need for dedicated research efforts to unlock the full potential of these immersive technologies. This review, therefore, outlines a roadmap for future research, encouraging innovation in visualization techniques, addressing identified challenges, and considering the ethical implications of VR and AI convergence.

Visualization of AI Systems in Virtual Reality: A Comprehensive Review

TL;DR

This paper surveys the visualization of AI systems in virtual reality by reviewing 18 peer‑reviewed studies up to January 2023. It analyzes how VR engines (notably Unity and Unreal) are used alongside AI backends (TensorFlow, PyTorch, etc.) to render and interact with neural networks and related models, while identifying challenges such as data dimensionality, cognitive load, and motion sickness. The authors synthesize visualization and interaction techniques (e.g., 3D network visualizations, feature-map views, immersive node-link diagrams) and discuss domain applications, implementation hurdles, and ethical considerations. The study provides a roadmap for future research, emphasizing more intuitive tools, standardization, cross‑tool comparability, and responsible integration of VR and AI technologies to unlock broader adoption and impact.

Abstract

This study provides a comprehensive review of the utilization of Virtual Reality (VR) for visualizing Artificial Intelligence (AI) systems, drawing on 18 selected studies. The results illuminate a complex interplay of tools, methods, and approaches, notably the prominence of VR engines like Unreal Engine and Unity. However, despite these tools, a universal solution for effective AI visualization remains elusive, reflecting the unique strengths and limitations of each technique. We observed the application of VR for AI visualization across multiple domains, despite challenges such as high data complexity and cognitive load. Moreover, it briefly discusses the emerging ethical considerations pertaining to the broad integration of these technologies. Despite these challenges, the field shows significant potential, emphasizing the need for dedicated research efforts to unlock the full potential of these immersive technologies. This review, therefore, outlines a roadmap for future research, encouraging innovation in visualization techniques, addressing identified challenges, and considering the ethical implications of VR and AI convergence.
Paper Structure (12 sections, 1 figure)

This paper contains 12 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Methods
  3. Search Strategy
  4. Inclusion and Exclusion Criteria
  5. Study Selection
  6. Data Extraction and Synthesis
  7. Study
  8. Results of Syntheses
  9. Virtual Reality and AI Systems
  10. Visualization and Interaction Techniques
  11. Application and Implementation
  12. Discussion and Conclusions

Figures (1)

  • Figure 1: Flow diagram of the literature search and selection process.