CardiacNet: Learning to Reconstruct Abnormalities for Cardiac Disease Assessment from Echocardiogram Videos

TL;DR

This work proposes a novel reconstruction-based approach named CardiacNet to learn a better representation of local cardiac structures and motion abnormalities through echocardiogram videos and proposes benchmark datasets named CardiacNet-PAH and CardiacNet-ASD to evaluate the effectiveness of cardiac disease assessment.

Abstract

Echocardiogram video plays a crucial role in analysing cardiac function and diagnosing cardiac diseases. Current deep neural network methods primarily aim to enhance diagnosis accuracy by incorporating prior knowledge, such as segmenting cardiac structures or lesions annotated by human experts. However, diagnosing the inconsistent behaviours of the heart, which exist across both spatial and temporal dimensions, remains extremely challenging. For instance, the analysis of cardiac motion acquires both spatial and temporal information from the heartbeat cycle. To address this issue, we propose a novel reconstruction-based approach named CardiacNet to learn a better representation of local cardiac structures and motion abnormalities through echocardiogram videos. CardiacNet is accompanied by the Consistency Deformation Codebook (CDC) and the Consistency Deformed-Discriminator (CDD) to learn the commonalities across abnormal and normal samples by incorporating cardiac prior knowledge. In addition, we propose benchmark datasets named CardiacNet-PAH and CardiacNet-ASD to evaluate the effectiveness of cardiac disease assessment. In experiments, our CardiacNet can achieve state-of-the-art results in three different cardiac disease assessment tasks on public datasets CAMUS, EchoNet, and our datasets. The code and dataset are available at: https://github.com/xmed-lab/CardiacNet.

Figures (6)

• Figure 1: Five examples from CardiacNet-PAH and CardiacNet-ASD datasets. The appearance between the Atrial Septal Defect (ASD) (a) and normal (b) is easy to distinguish. For (c), (d) and (e), the appearance of cardiac structures in the Pulmonary Arterial Hypertension patient (c) and normal (d) are similar. In contrast, normal cases (d) and (e) show significant differences. Indicates that using a single image is not able to diagnose this type of cardiac disease. Clinically, experienced physicians will use echocardiogram videos with cardiac motion information to make diagnoses.
• Figure 2: The overview of our CardiacNet, sample normal case $X$ and abnormal case $Y$, reconstruct the corresponding abnormal and normal results through networks $\phi^A(\cdot)$ and $\phi^B(\cdot)$, respectively. The Consistency Deformation Discriminator (CDD) is introduced to retain high reconstruction quality and allow the reconstruction results to be consistent with actual cases.
• Figure 3: The description of the one-way process of our CardiacNet. The encoded feature $F$ of the normal case will be quantized by the deformation codebook $Z$ as quantized feature $\tilde{F}$. The decoder then recovers $\tilde{F}$ to the reconstructed abnormal result. Accompanied by the abnormal case sampled from the dataset, the Consistency Deformation Discriminator (CDD) is introduced to improve the consistency between reconstructed results and actual samples with regional discrimination.
• Figure 4: The optimal transport distance optimization between two networks $\phi^A(\cdot)$ and $\phi^B(\cdot)$. Memory banks $\mathcal{M}^A$ and $\mathcal{M}^B$ store the features of normal and abnormal data samples, respectively. The loss $\mathcal{L}_\text{OT}(\mathcal{M}^A,\mathcal{M}^B)$ makes these two distributions keep away from each other. Losses $\mathcal{L}_\text{dis}(\tilde{F}_{X},\overline{\mathcal{M}}^A)$, and $\mathcal{L}_\text{dis}(\tilde{F}_Y,\overline{\mathcal{M}}^B)$ make representations of clusters more consistent.
• Figure 5: The visualization of t-SNE results between learned embedding of normal and abnormal cases by our CardiacNet in epochs 10, 100, 300 and 500.