Ascribe New Dimensions to Scientific Data Visualization with VR

TL;DR

The paper addresses the limitation of 2D interaction in exploring complex scientific data and introduces ASCRIBE-VR, a VR platform built on Unreal Engine that integrates AI-driven segmentation, data filtering, quantification, immersion, and interaction to create digital twins for materials and biomedical imaging. By leveraging VR on Meta Quest 3 and a suite of AI techniques (CNNs, Vision Transformers, Gaussian Processes) within a human-in-the-loop framework, it demonstrates immersive, multisensory analysis and real-time data fusion. The work contrasts ASCRIBE-VR with existing VR tools, highlighting its unique capabilities in texture editing, image visualization, multi-object manipulation, mesh import, and multiplayer collaboration. This platform aims to accelerate scientific discovery by blending automated analysis with immersive exploration, enabling more intuitive validation and cross-disciplinary collaboration in large-scale 3D datasets.

Abstract

For over half a century, the computer mouse has been the primary tool for interacting with digital data, yet it remains a limiting factor in exploring complex, multi-scale scientific images. Traditional 2D visualization methods hinder intuitive analysis of inherently 3D structures. Virtual Reality (VR) offers a transformative alternative, providing immersive, interactive environments that enhance data comprehension. This article introduces ASCRIBE-VR, a VR platform of Autonomous Solutions for Computational Research with Immersive Browsing \& Exploration, which integrates AI-driven algorithms with scientific images. ASCRIBE-VR enables multimodal analysis, structural assessments, and immersive visualization, supporting scientific visualization of advanced datasets such as X-ray CT, Magnetic Resonance, and synthetic 3D imaging. Our VR tools, compatible with Meta Quest, can consume the output of our AI-based segmentation and iterative feedback processes to enable seamless exploration of large-scale 3D images. By merging AI-generated results with VR visualization, ASCRIBE-VR enhances scientific discovery, bridging the gap between computational analysis and human intuition in materials research, connecting human-in-the-loop with digital twins.

Figures (10)

• Figure 1: Virtual Menu of functionalities. More details in the text.
• Figure 2: Illustration of Virtual Menu locomotion (left) and the teleport function (right). After enabling this feature in the Main menu, the user can move through the virtual environment by teleporting, guided by a light signal on the floor that indicates where they will be teleported.
• Figure 3: Illustration of the Virtual Menu (Left) and the Push and Pull function (Right). After enabling the Push and Pull functionality, the user can move objects closer or farther away.
• Figure 4: Illustration of the Resize Mesh function in the virtual environment. With this functionality, the user can increase or decrease the size of objects as they move their hands closer or further apart.
• Figure 5: Illustration of the process of importing a virtual mesh into the application's virtual environment. In image 1, we have an interface that shows all the files available to be imported. In image 2, we have the interface after selecting a file and the Import Selected File option.