Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multimodal Learning To Improve Cardiac Late Mechanical Activation Detection From Cine MR Images

Jiarui Xing, Nian Wu, Kenneth Bilchick, Frederick Epstein, Miaomiao Zhang

TL;DR

The paper tackles the challenge of detecting late mechanical activation (LMA) from routinely acquired cine CMR by leveraging DENSE-derived myocardial strains as supervision. It introduces a multimodal, joint learning framework with a registration-guided strain network and an LMA regression network that are trained end-to-end to predict both strains and the onset of circumferential shortening (TOS) from cine data. By basing the strain predictions on latent motion features learned from cine images and guiding them with DENSE data, the approach achieves substantial improvements in LMA detection and yields activation maps that better approximate DENSE-based analyses. This work enhances the practicality of high-fidelity strain analysis in settings with limited access to DENSE, paving the way for broader validation and deployment across diverse patient cohorts.

Abstract

This paper presents a multimodal deep learning framework that utilizes advanced image techniques to improve the performance of clinical analysis heavily dependent on routinely acquired standard images. More specifically, we develop a joint learning network that for the first time leverages the accuracy and reproducibility of myocardial strains obtained from Displacement Encoding with Stimulated Echo (DENSE) to guide the analysis of cine cardiac magnetic resonance (CMR) imaging in late mechanical activation (LMA) detection. An image registration network is utilized to acquire the knowledge of cardiac motions, an important feature estimator of strain values, from standard cine CMRs. Our framework consists of two major components: (i) a DENSE-supervised strain network leveraging latent motion features learned from a registration network to predict myocardial strains; and (ii) a LMA network taking advantage of the predicted strain for effective LMA detection. Experimental results show that our proposed work substantially improves the performance of strain analysis and LMA detection from cine CMR images, aligning more closely with the achievements of DENSE.

Multimodal Learning To Improve Cardiac Late Mechanical Activation Detection From Cine MR Images

TL;DR

The paper tackles the challenge of detecting late mechanical activation (LMA) from routinely acquired cine CMR by leveraging DENSE-derived myocardial strains as supervision. It introduces a multimodal, joint learning framework with a registration-guided strain network and an LMA regression network that are trained end-to-end to predict both strains and the onset of circumferential shortening (TOS) from cine data. By basing the strain predictions on latent motion features learned from cine images and guiding them with DENSE data, the approach achieves substantial improvements in LMA detection and yields activation maps that better approximate DENSE-based analyses. This work enhances the practicality of high-fidelity strain analysis in settings with limited access to DENSE, paving the way for broader validation and deployment across diverse patient cohorts.

Abstract

This paper presents a multimodal deep learning framework that utilizes advanced image techniques to improve the performance of clinical analysis heavily dependent on routinely acquired standard images. More specifically, we develop a joint learning network that for the first time leverages the accuracy and reproducibility of myocardial strains obtained from Displacement Encoding with Stimulated Echo (DENSE) to guide the analysis of cine cardiac magnetic resonance (CMR) imaging in late mechanical activation (LMA) detection. An image registration network is utilized to acquire the knowledge of cardiac motions, an important feature estimator of strain values, from standard cine CMRs. Our framework consists of two major components: (i) a DENSE-supervised strain network leveraging latent motion features learned from a registration network to predict myocardial strains; and (ii) a LMA network taking advantage of the predicted strain for effective LMA detection. Experimental results show that our proposed work substantially improves the performance of strain analysis and LMA detection from cine CMR images, aligning more closely with the achievements of DENSE.
Paper Structure (5 sections, 2 equations, 4 figures)

This paper contains 5 sections, 2 equations, 4 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Our Multimodal Learning Network
  4. Experiments
  5. Conclusion

Figures (4)

  • Figure 1: The multimodal joint learning framework with registration-guided strain prediction and LMA detection networks.
  • Figure 2: Example of (a) temporal CMRs overlaid with displacement fields; (b) LV strain (contraction/stretching in blue/red; the blue circle shows the sampling starting location); and (c) 2D strain matrix and its corresponding TOS curve.
  • Figure 3: Top to bottom panel: a comparison of TOS prediction from all methods vs. manually labeled TOS (marked in solid black) overlaid on strain matrix; TOS regression mean square error vs. LMA classification accuracy from all methods.
  • Figure 4: Left to right: a comparison of 3D Activation Maps from ground truth vs. reconstructed from all methods.