S2MNet: Speckle-To-Mesh Net for Three-Dimensional Cardiac Morphology Reconstruction via Echocardiogram

TL;DR

This work tackles the challenge of reconstructing high-fidelity 3D cardiac morphology from limited 2D echocardiograms. It introduces S2MNet, a template-based deformation-field approach that learns to predict continuous vector fields from echo-styled images, leveraging synthetic six-view data generated from a 3D heart mesh and CycleGAN-style domain adaptation. The method yields accurate 3D reconstructions (low $MSE$ and competitive IoU) and demonstrates clinical relevance by correlating left ventricular EF derived from the 3D meshes with GLPS measurements ($PCC = -0.82$). The framework offers a practical path toward real-time, patient-specific 3D cardiac visualization from standard echocardiography, with potential to improve functional assessment and monitoring in clinical settings.

Abstract

Echocardiogram is the most commonly used imaging modality in cardiac assessment duo to its non-invasive nature, real-time capability, and cost-effectiveness. Despite its advantages, most clinical echocardiograms provide only two-dimensional views, limiting the ability to fully assess cardiac anatomy and function in three dimensions. While three-dimensional echocardiography exists, it often suffers from reduced resolution, limited availability, and higher acquisition costs. To overcome these challenges, we propose a deep learning framework S2MNet that reconstructs continuous and high-fidelity 3D heart models by integrating six slices of routinely acquired 2D echocardiogram views. Our method has three advantages. First, our method avoid the difficulties on training data acquasition by simulate six of 2D echocardiogram images from corresponding slices of a given 3D heart mesh. Second, we introduce a deformation field-based method, which avoid spatial discontinuities or structural artifacts in 3D echocardiogram reconstructions. We validate our method using clinically collected echocardiogram and demonstrate that our estimated left ventricular volume, a key clinical indicator of cardiac function, is strongly correlated with the doctor measured GLPS, a clinical measurement that should demonstrate a negative correlation with LVE in medical theory. This association confirms the reliability of our proposed 3D construction method.

Figures (5)

• Figure 1: An overview of our 3D cardiac morphology reconstruction framework. In stage 1, synthetic 2D cardiac image slices are generated from a 3D heart model. In stage 2, these binary images are preprocessed and passed through a pretrained CycleGAN to their similarity to real echocardiogram images. In stage 3, a vector field is computed between a template heart mesh and other cardiac meshes through deformation method. In stage 4, S2MNet is trained to predict vector fields from ultrasound-styled images, which are added to the template mesh to reconstruct the 3D cardiac structure.
• Figure 2: An overview of our S2MNet for 3D reconstruction. The encoder extracts 2D features, while the decoder and trillinear layer generates coarse vector fields. A weight generator assigns weights to each coarse vector field, which are fused via weighted summation to produce the final vector field. The output is a deformation vector which should be added to the template to get 3D heart mesh.
• Figure 3: A box-and-whisker plot for IoU comparison of different reconstruction methods at a resolution of $128^3$. Pi2Vox and E-Pi2Vox refer to methods proposed in pix2voxe-p2vforheart. "Fast" and "Acc" indicate different network architectures, with "Fast" using fewer layers for faster inference. Our method is shown in red on the left.
• Figure 4: 3D reconstruction result. The red mesh is the target (available from synthetic data), the light blue is the template, and the dark blue is our S2MNet prediction. In the low-MSE case, prediction and target overlap almost perfectly (appearing purple). Even in the high-MSE case, alignment remains strong with only slight visible discrepancies.
• Figure 5: Segmentation result: the light blue area represents the segmentation mask, which is obtained by slicing our predicted mesh along the predicted slicing angle. The mask aligns perfectly with the tissue in the echocardiogram images.