EchoSim4D: A Proof-of-Concept Gamified XR Echocardiography Training Simulator for Neonates using 4D Ultrasound Volume

TL;DR

This work tackles the shortage of neonatal 4D echocardiography training tools by introducing EchoSim4D, an XR-based simulator that renders 4D ultrasound volumes in Unity and provides a low-end optimized mode. It combines a sensorized neonatal manikin and a 3D-printed transducer to simulate realistic probe maneuvers and offers a gamified two-level curriculum with real-time feedback. A data-pipeline converts clinical 4D DICOM data into Unity-friendly formats (NRRD/GIF) to support both real-time and preloaded visualization, enabling scalable training. Validation with 10 postgraduate trainees showed strong visual fidelity and educational value, while revealing ergonomic and hardware optimization needs to advance clinical skill transfer.

Abstract

Neonatal echocardiography is vital for early detection of heart anomalies in newborns, enabling timely, non-invasive interventions where 4D ultrasound, adds the dimension of time to 3D imaging, enhances diagnostic capabilities by visualizing real-time heart dynamics. However, training for 4D neonatal echocardiography is limited by the lack of simulators that support 4D Ultrasound volume visualization within gamified environments. This paper introduces EchoSim4D, an XR-based simulator leveraging novel pipeline for visualizing 4D volume data in Unity, incorporating real-time volume reconstruction, and a preloaded version optimized for low-end systems. EchoSim4D integrates a sensor-equipped manikin and a custom 3D-printed transducer with a 6-DOF sensor, replicating the precise probe maneuvers necessary for neonatal echocardiography. In a validation study with postgraduate medical students (0-5 years of experience), supervised by a domain expert, EchoSim4D demonstrated high visual fidelity and training efficacy. Findings suggest that 4D visualization techniques hold significant potential for advancing medical training in neonatal echocardiography.

Figures (10)

• Figure 1: Standard Probe Maneuvers: a) Sliding, b) Rotating, c) Tilting, d) Rocking, e) Pressure, and f) Sweeping
• Figure 1: Standard Transthoracic Echocardiographic Views
• Figure 2: Clock System
• Figure 3: Standard TTE Locations
• Figure 4: a) 4D Volume Data Acquisition, b) EchoSim4D Hardware