Enabling Additive Manufacturing Part Inspection of Digital Twins via Collaborative Virtual Reality

TL;DR

The paper addresses the challenge of inspecting multimodal digital twins in additive manufacturing by introducing a collaborative virtual reality framework that supports multimodal alignment, occlusion-aware volumetric visualization, streaming of large CT volumes, and real-time team collaboration. Implemented as a Unity-based client–server system with OpenViSUS data streaming and networked synchronization, it enables distributed experts to jointly inspect AM parts and processes in an immersive environment. Expert evaluations and a real-data case study demonstrate improved inspection capabilities, guidance for future metadata and measurement tools, and the potential for training and knowledge transfer. This work advances AM DT inspection by enabling immersive, scalable, and collaborative exploration of complex, data-rich DT representations, establishing a new benchmark for VR-enabled DT analysis in manufacturing.

Abstract

Digital twins (DTs) are an emerging capability in additive manufacturing (AM), set to revolutionize design optimization, inspection, in situ monitoring, and root cause analysis. AM DTs typically incorporate multimodal data streams, ranging from machine toolpaths and in-process imaging to X-ray CT scans and performance metrics. Despite the evolution of DT platforms, challenges remain in effectively inspecting them for actionable insights, either individually or in a multidisciplinary team setting. Quality assurance, manufacturing departments, pilot labs, and plant operations must collaborate closely to reliably produce parts at scale. This is particularly crucial in AM where complex structures require a collaborative and multidisciplinary approach. Additionally, the large-scale data originating from different modalities and their inherent 3D nature pose significant hurdles for traditional 2D desktop-based inspection methods. To address these challenges and increase the value proposition of DTs, we introduce a novel virtual reality (VR) framework to facilitate collaborative and real-time inspection of DTs in AM. This framework includes advanced features for intuitive alignment and visualization of multimodal data, visual occlusion management, streaming large-scale volumetric data, and collaborative tools, substantially improving the inspection of AM components and processes to fully exploit the potential of DTs in AM.

Figures (6)

• Figure 1: Overview of the proposed framework aimed to enhance AM part inspection of DTs via collaborative VR. It supports multimodal data alignment and visualization, streaming large-scale and multi-resolution volumetric data, visual occlusion management, as well as team collaboration features. Furthermore, it allows multiple users either co-located or remote to collaborate in a shared virtual environment.
• Figure 2: Multimodal data visualization to support the process of data inspection and analysis: (a) users can load and explore data from multiple data streams in the virtual environment, (b) data representations based on printed layer, including prescribed toolpath (green), machine toolpath (yellow), and error (red), (c) users can enable in-process imaging to further inspect the data, and (d) volumetric data visualization from X-ray CT scans with machine toolpaths.
• Figure 3: Occlusion management with volumetric rendering in immersive VR. We implemented the colorized volume with the transfer functions and color maps. With the cutting objects, it allows users to explore the inner structures of the volumetric data smoothly, e.g., using a cross-section plane (a), sphere (b and c), or box (d and e) cutout with inclusive and exclusive modes, respectively.
• Figure 4: Various features for data exploration and inspection as well as related features to improve team-based collaboration: those features include adjusting CT window width/level, using cross-section plane and axis slice view (axial, coronal, and sagittal) to explore and inspect data, drawing annotations, aligning and visualizing multimodal data, streaming multi-resolution CT volume, loading and interacting 3D scanned models, and drawing on the virtual whiteboard.
• Figure 5: Multi-user collaboration for AM data exploration and inspection: (a) collaborative users virtually join and explore AM data, i.e., X-ray CT volume, in a shared virtual environment, and (b) users can inspect further details of the data using various inspection features, i.e., a cross-section plane. The interactions between users are synchronized in real time.