Explicit Differentiable Slicing and Global Deformation for Cardiac Mesh Reconstruction

TL;DR

A novel explicit differentiable voxelization and slicing (DVS) algorithm that allows gradient backpropagation to a mesh from its slices, facilitating refined mesh optimization directly supervised by the losses defined on 2D images is proposed.

Abstract

Mesh reconstruction of the cardiac anatomy from medical images is useful for shape and motion measurements and biophysics simulations to facilitate the assessment of cardiac function and health. However, 3D medical images are often acquired as 2D slices that are sparsely sampled and noisy, and mesh reconstruction on such data is a challenging task. Traditional voxel-based approaches rely on pre- and post-processing that compromises image fidelity, while mesh-level deep learning approaches require mesh annotations that are difficult to get. Therefore, direct cross-domain supervision from 2D images to meshes is a key technique for advancing 3D learning in medical imaging, but it has not been well-developed. While there have been attempts to approximate the optimized meshes' slicing, few existing methods directly use 2D slices to supervise mesh reconstruction in a differentiable manner. Here, we propose a novel explicit differentiable voxelization and slicing (DVS) algorithm that allows gradient backpropagation to a mesh from its slices, facilitating refined mesh optimization directly supervised by the losses defined on 2D images. Further, we propose an innovative framework for extracting patient-specific left ventricle (LV) meshes from medical images by coupling DVS with a graph harmonic deformation (GHD) mesh morphing descriptor of cardiac shape that naturally preserves mesh quality and smoothness during optimization. Experimental results demonstrate that our method achieves state-of-the-art performance in cardiac mesh reconstruction tasks from CT and MRI, with an overall Dice score of 90% on multi-datasets, outperforming existing approaches. The proposed method can further quantify clinically useful parameters such as ejection fraction and global myocardial strains, closely matching the ground truth and surpassing the traditional voxel-based approach in sparse images.

Figures (8)

• Figure 1: The Motivation and Novelty of our Proposed Method. Block A depicts the image-level learning and post-processing for 3D mesh reconstruction. Block B shows previous mesh-level optimization approaches with vertex-wise deformation and mesh ground truth (GT) supervision. Block C introduces our proposed method, which incorporates differentiable mesh operations and global mesh deformation based on graph harmonics (GHD), enabling direct supervision between meshes and images. Green marks indicate the advantages of methods, red marks indicate the weaknesses. The blue arrows highlight the key technical contributions of our work: differentiable voxelization, slicing (DVS) and the global deformation GHD.
• Figure 2: The inverse quadratic fields derived from query points (a) outside and (b) inside a left ventricle mesh. The length of the arrows represents the field strength, and the color represents the inner product of the field and the outward normal vector of the mesh, blue signifies a negative value while red signifies a positive value.
• Figure 3: The energy strips of the mixed Laplacian. The eigenvalues of the mixed Laplacian compose different energy strips rather than merely rank down to up. The higher energy strips represent the high-frequency components, which require more degrees of freedom to represent the mesh deformation. The Redshifts on each deformed LV mesh represent the positive (outward) displacements, and the blue for the negative (inward).
• Figure 4: The pipeline of the 3D mesh reconstruction of the left ventricle from MRI images. The pipeline starts from the segmentation of medical images, followed by a rigid orientation to align the canonical shape to the target roughly. The GHD optimization supervised by the differentiable slicing on the images yields the reconstructed mesh. Auxiliary supervisions like thickness and volume are alternatively applied to improve the mesh quality for better biomedical plausibility.
• Figure 5: The convergence of GHD compared to the traditional vertex-wise mesh morphing method, demonstrating that GHD converges faster and more robustly. The zoomed-in view of the mesh shows that the GHD method preserves the triangle quality and smoothness during the optimization, while the vertex-wise deformation method has irregular triangles and sharp edges during optimization.