Neural Implicit Heart Coordinates: 3D cardiac shape reconstruction from sparse segmentations

TL;DR

The paper tackles the challenge of reconstructing patient-specific 3D cardiac anatomy from sparse CMR segmentations. It introduces Neural Implicit Heart Coordinates (NIHCs), a standardized implicit reference frame derived from CobivecoX Universal Ventricular Coordinates, enabling cross-subject, high-fidelity reconstructions. A joint neural implicit framework (SEG-NIF and UVC-NIF) with a shared latent prior yields dense segmentations and 3D meshes at arbitrary resolutions from limited input data, dramatically reducing inference time from ~60 s to 5–15 s. Evaluations on a large UK Biobank-derived cohort show robust performance in both diseased and healthy populations, with mean surface errors around $2.3$–$2.51$ mm and Dice scores above 0.9 for the LV myocardium, though valve-plane regions and slice misalignment remain challenging. This approach advances clinical workflows for digital twins and patient-specific cardiac simulations by enabling fast, coherent, and scalable 3D heart reconstructions from sparse inputs.

Abstract

Accurate reconstruction of cardiac anatomy from sparse clinical images remains a major challenge in patient-specific modeling. While neural implicit functions have previously been applied to this task, their application to mapping anatomical consistency across subjects has been limited. In this work, we introduce Neural Implicit Heart Coordinates (NIHCs), a standardized implicit coordinate system, based on universal ventricular coordinates, that provides a common anatomical reference frame for the human heart. Our method predicts NIHCs directly from a limited number of 2D segmentations (sparse acquisition) and subsequently decodes them into dense 3D segmentations and high-resolution meshes at arbitrary output resolution. Trained on a large dataset of 5,000 cardiac meshes, the model achieves high reconstruction accuracy on clinical contours, with mean Euclidean surface errors of 2.51$\pm$0.33 mm in a diseased cohort (n=4549) and 2.3$\pm$0.36 mm in a healthy cohort (n=5576). The NIHC representation enables anatomically coherent reconstruction even under severe slice sparsity and segmentation noise, faithfully recovering complex structures such as the valve planes. Compared with traditional pipelines, inference time is reduced from over 60 s to 5-15 s. These results demonstrate that NIHCs constitute a robust and efficient anatomical representation for patient-specific 3D cardiac reconstruction from minimal input data.

Figures (17)

• Figure 1: Overview of our pipeline for simultaneous prediction of dense segmentations and 3D personalized meshes from sparse segmentations. During inference, first, coordinates from known points (e.g., segmentation contours) are used together with a previously trained MLP model (MLP$_{seg}$) to predict occupancy at those points and optimize a latent shape prior vector (see red dotted arrows). This process is referred to as inference optimization. Once the optimal latent shape vector has been found, a second MLP (MLP$_{reg}$) is used together with queried Universal Ventricular Coordinates (UVCs) to predict a 3D personalized mesh. The optimal latent shape prior vector can also be used by the MLP$_{seg}$ to generate a dense segmentation. As both MLPs learn a continuous mapping from their respective inputs to outputs, both dense segmentation maps and personalized 3D models can be generated at any arbitrary resolution.
• Figure 2: Data generation pipeline. First, a dataset of image-derived contours (referred to as SEG dataset) was created by extracting contours (using Circle cvi42 postprocessing software Version 5.11 1505). This dataset was used during testing. Second, a mesh-derived dataset (referred to as SYN) was generated by fitting a template mesh to each case via a diffeomorphic fitting method mauger2018iterative, after having corrected for breath-hold misalignment following sinclair2017fully. After alignment, an average UKBB shape was computed, and Universal Ventricular Coordinates (UVCs) were calculated using the CovibecoX method pankewitz2024universal and transferred to each mesh using point-to-point correspondences. The final SYN dataset consists of synthetic contours generated by slicing these meshes and assigning labels based on the UVCs. The SYN dataset was used during both training and testing.
• Figure 3: Overview of the proposed pipeline. During training, $MLP_{\text{seg}}$ and $MLP_{\text{reg}}$ are jointly optimized using a combined loss for the segmentation and regression tasks. Backpropagation updates the weights of both MLPs as well as the learnable latent prior $h$. During inference optimization, only $MLP_{\text{seg}}$ is used (kept frozen) to refine the latent prior based on the segmentation task. Finally, during prediction, both MLPs are frozen, and query points (in Cartesian or UVC coordinates) are used to reconstruct either a label map or a 3D mesh.
• Figure 4: Plot of the Euclidean distance calculated from the reference (REF) and segmentation (SEG) meshes to the SEG contours obtained from the CMR slices.
• Figure 5: Comparison of ED values between REF meshes and SYN/SEG predictions in healthy and diseased samples. Violin plots show the distribution of values, while boxplots indicate medians and interquartile ranges.