Learning Volumetric Neural Deformable Models to Recover 3D Regional Heart Wall Motion from Multi-Planar Tagged MRI

TL;DR

This work addresses the challenge of recovering dense 3D regional heart wall motion from sparse 2D SPAMM-tagged MRI cues. It introduces volumetric neural deformable models ($\upsilon$NDMs) that combine global deformation parameter functions with a diffeomorphic local flow, learned via a hybrid point transformer and neural ODEs, to map apparent motion into true 3D motion. A large-scale synthetic data generation pipeline based on 3D CMR data enables supervised training and quantitative evaluation of motion recovery. Results show improved accuracy and topology, with substantially faster inference compared to traditional iterative deformable models, indicating strong potential for broader application in time-resolved cardiac MRI.

Abstract

Multi-planar tagged MRI is the gold standard for regional heart wall motion evaluation. However, accurate recovery of the 3D true heart wall motion from a set of 2D apparent motion cues is challenging, due to incomplete sampling of the true motion and difficulty in information fusion from apparent motion cues observed on multiple imaging planes. To solve these challenges, we introduce a novel class of volumetric neural deformable models ($\upsilon$NDMs). Our $\upsilon$NDMs represent heart wall geometry and motion through a set of low-dimensional global deformation parameter functions and a diffeomorphic point flow regularized local deformation field. To learn such global and local deformation for 2D apparent motion mapping to 3D true motion, we design a hybrid point transformer, which incorporates both point cross-attention and self-attention mechanisms. While use of point cross-attention can learn to fuse 2D apparent motion cues into material point true motion hints, point self-attention hierarchically organised as an encoder-decoder structure can further learn to refine these hints and map them into 3D true motion. We have performed experiments on a large cohort of synthetic 3D regional heart wall motion dataset. The results demonstrated the high accuracy of our method for the recovery of dense 3D true motion from sparse 2D apparent motion cues. Project page is at https://github.com/DeepTag/VolumetricNeuralDeformableModels.

Figures (11)

• Figure 1: (a) Bi-ventricle heart model. LV: left ventricle. RV: right ventricle. Along the LV long-axis, the heart wall is divided into three parts: basal, middle and apical regions. (b) Long-axis (LAX) (top) and short-axis (SAX) (bottom) imaging planes. (c) Multi-planar tagged MRI sequences of the heart. The number under an image indicates percentage of a cardiac cycle. ED: end diastole. ES: end systole. Myo: myocardium.
• Figure 2: (a) Material points $\mathbf{M}$ and SPAMM datapoints $\mathbf{S}$. (b) In-plane ($x$ and $y$) apparent motion cues (red arrow) provided by two corresponding SAX SPAMM datapoints (yellow) and through-plane ($z$) apparent motion cue (red arrow) provided by two corresponding LAX SPAMM datapoints (yellow). The dashed lines in the mesh $\mathbf{M}(t_{1})$ show the SAX and LAX imaging planes identical to those in the mesh $\mathbf{M}(t_{0})$. 'r': radius direction.
• Figure 3: (a) Coordinate system definition of a volumetric deformable model. (b) Short-axis view. (c) Long-axis view. (d) Twisting deformation $\tau$ that rotates an in-plane point from $\boldsymbol{p}$ to $\boldsymbol{p}'$.
• Figure 4: Steps for heart wall geometry and motion simulation, and SPAMM datapoints computation. We first fit the inner and outer wall at (a) ED and (b) ES from a sparse point cloud, using a two-layer $\upsilon$NDM in step $1$, and then interpolate middle wall layers in step $2$. We synthesize missing heart wall models at other temporal points (e.g., (c) systole and (d) diastole) by interpolating the models at ED and ES phases in step $3$. In step $4$, we compute the SPAMM datapoints in (e) SAX and (f) LAX views using a mesh-plane clipping algorithm.
• Figure 5: Heart wall 3D motion recovery network. The point cross-attention layer fuses sparse apparent motion cues $\mathbf{A}(t_{q}, t_{q+1}) = \left\{\mathbf{S}(t_{q}), \mathbf{S}(t_{q+1})\right\}$ into true motion hints $\mathbf{H}$. Multiple point self-attention layers transform true motion hints $\mathbf{H}$ to latent motion code $\boldsymbol{z}$, which is further used by the MLPs and conditional neural ordinary diffeomorphic blocks (NODEs) to predict 3D true motion of material points. In the dashed box, pink points show $\mathbf{S}(t_{q})$; purple points show $\mathbf{S}(t_{q+1})$; lines between $\mathbf{S}(t_{q})$ and $\mathbf{S}(t_{q+1})$ show one-to-one correspondences. Yellow points on/near the mesh of the middle myocardium wall layer show ground truth material points.