Multi-view Hybrid Graph Convolutional Network for Volume-to-mesh Reconstruction in Cardiovascular MRI

TL;DR

HybridVNet introduces a direct image-to-mesh framework for cardiovascular MRI by fusing multi-view volumetric encoding with a spectral graph decoder to produce both surface and tetrahedral meshes. The model uses a variational encoder, deep supervision across graph resolutions, and specialized regularisation for tetrahedral elements, achieving state-of-the-art performance on UK Biobank data and outperforming segmentation-to-mesh pipelines in both accuracy and speed. Multi-view fusion (short-axis and long-axis views) improves mesh fidelity across cardiac structures, including regions not well-captured in SAX alone, with clinically validated volume measurements showing strong agreement with expert annotations. The approach significantly accelerates mesh generation (forward pass times as low as 0.04 seconds on GPUs) and offers robust performance for large-scale computational anatomy studies, while highlighting trade-offs between mesh quality and anatomical accuracy through tetrahedral regularisation. These contributions advance automated, high-fidelity cardiac modeling for simulations and biomarker discovery, with potential extensions to other organs and topology-flexible mesh generation.

Abstract

Cardiovascular magnetic resonance imaging is emerging as a crucial tool to examine cardiac morphology and function. Essential to this endeavour are anatomical 3D surface and volumetric meshes derived from CMR images, which facilitate computational anatomy studies, biomarker discovery, and in-silico simulations. Traditional approaches typically follow complex multi-step pipelines, first segmenting images and then reconstructing meshes, making them time-consuming and prone to error propagation. In response, we introduce HybridVNet, a novel architecture for direct image-to-mesh extraction seamlessly integrating standard convolutional neural networks with graph convolutions, which we prove can efficiently handle surface and volumetric meshes by encoding them as graph structures. To further enhance accuracy, we propose a multi-view HybridVNet architecture which processes both long axis and short axis CMR, showing that it can increase the performance of cardiac MR mesh generation. Our model combines traditional convolutional networks with variational graph generative models, deep supervision and mesh-specific regularisation. Experiments on a comprehensive dataset from the UK Biobank confirm the potential of HybridVNet to significantly advance cardiac imaging and computational cardiology by efficiently generating high-fidelity meshes from CMR images. Multi-view HybridVNet outperforms the state-of-the-art, achieving improvements of up to $\sim$27\% reduction in Mean Contour Distance (from 1.86 mm to 1.35 mm for the LV Myocardium), up to $\sim$18\% improvement in Hausdorff distance (from 4.74 mm to 3.89mm, for the LV Endocardium), and up to $\sim$8\% in Dice Coefficient (from 0.78 to 0.84, for the LV Myocardium), highlighting its superior accuracy.

Figures (9)

• Figure 1: Mesh generation pipelines
• Figure 2: Multi-view HybridVNet model architecture: The proposed model uses a variational encoder-decoder architecture to generate a graph representation of a desired organ from multi-view input images. The encoder consists of independent branches for each input view, concatenated to obtain a joint latent space. The latent code is then passed through a fully connected layer and reshaped to obtain the initial node features for the graph convolutional decoder. This decoder uses the initial node features to generate a final graph representation of the organ.
• Figure 3: Qualitative performance evaluation of MV-HybridVNet (with $\lambda_{lap}=0.01$) on cardiac MRI segmentation across six test subjects, showing the predictions against ground truth. For each subject, we present mesh-based visualizations (left three columns) showing the predicted surface, ground-truth surface, and their error map comparison, alongside 2D visualizations (right three columns) at mid-ventricular level displaying the original middle slice MRI, predicted segmentation overlay, and ground-truth segmentation overlay. The top four rows demonstrate the performance on healthy subjects, while the bottom two rows showcase segmentation results from subjects with myocardial infarction.
• Figure 4: Average segmentation error (measured as MAE) across the 17 AHA left-ventricular segments on the test set of 600 subjects, evaluated at end-diastole (ED) and end-systole (ES). A consistently higher error is observed for ES segmentation, with slightly increased errors around the antero-lateral region in both phases. The table shows the values per segment as average $\pm$ standard deviation.
• Figure 5: Qualitative analysis of the impact of Laplacian regularisation term on surface mesh smoothness. It demonstrates the influence of adjusting the regularisation parameter on mesh quality. The best quantitative results regarding MSE for the validation split were achieved when $\lambda_{lap} = 0.01$.