Multimodal Variational Autoencoder for Low-cost Cardiac Hemodynamics Instability Detection

TL;DR

The study addresses CHDI detection by predicting $PAWP$ from inexpensive CXR and ECG data. It introduces CardioVAE_X,G, a multimodal variational autoencoder that uses a tri-stream pre-training strategy to learn both shared and modality-specific representations from a large unlabeled dataset (MIMIC) and then fine-tunes on a smaller labeled ASPIRE cohort ($50{,}982$ unlabeled pairs; $795$ labeled subjects). Results show competitive performance with an overall $AUROC$ of about $0.79$ and accuracy of about $0.77$, while unimodal analyses and interpretability via integrated gradients support clinical applicability. This approach demonstrates that combining low-cost modalities with unsupervised pre-training can approach MRI-based performance and yields interpretable insights to aid decision-making in critical care.

Abstract

Recent advancements in non-invasive detection of cardiac hemodynamic instability (CHDI) primarily focus on applying machine learning techniques to a single data modality, e.g. cardiac magnetic resonance imaging (MRI). Despite their potential, these approaches often fall short especially when the size of labeled patient data is limited, a common challenge in the medical domain. Furthermore, only a few studies have explored multimodal methods to study CHDI, which mostly rely on costly modalities such as cardiac MRI and echocardiogram. In response to these limitations, we propose a novel multimodal variational autoencoder ($\text{CardioVAE}_\text{X,G}$) to integrate low-cost chest X-ray (CXR) and electrocardiogram (ECG) modalities with pre-training on a large unlabeled dataset. Specifically, $\text{CardioVAE}_\text{X,G}$ introduces a novel tri-stream pre-training strategy to learn both shared and modality-specific features, thus enabling fine-tuning with both unimodal and multimodal datasets. We pre-train $\text{CardioVAE}_\text{X,G}$ on a large, unlabeled dataset of $50,982$ subjects from a subset of MIMIC database and then fine-tune the pre-trained model on a labeled dataset of $795$ subjects from the ASPIRE registry. Comprehensive evaluations against existing methods show that $\text{CardioVAE}_\text{X,G}$ offers promising performance (AUROC $=0.79$ and Accuracy $=0.77$), representing a significant step forward in non-invasive prediction of CHDI. Our model also excels in producing fine interpretations of predictions directly associated with clinical features, thereby supporting clinical decision-making.

Figures (2)

• Figure 1: The baselines and proposed $\text{CardioVAE}_\text{X,G}$ for PAWP prediction. (a) Top left: the single stream MVAE baselines wu2018multimodalli2023towards require pre-training on paired data (CXR, ECG) and utilize only the shared features using concatenation or product of expert (PoE) hinton2002training based multimodal fusion methods. (b) Bottom: Tri-stream multimodal pre-training that can learn both shared and modality-specific features. (c) Top right: fine-tuning on ASPIRE registry. Due to the tri-stream flow, our model can be fine-tuned on a single modality or both modalities.
• Figure 2: Interpretability of $\text{CardioVAE}_\text{X,G}$ for two subjects using integrated gradients method sundararajan2017axiomatic. (a) 1D ECG (top left) and CXR (bottom left) for normal PAWP, (b) 1D ECG (top right) and CXR (bottom right) for elevated PAWP. Green annotations on CXRs highlight seven regions of the heart and lungs, marked by an expert clinician for enhanced visualization of key areas. The 1D ECG signal was smoothed with NeuroKit2 makowski2021neurokit2 library for better visualization.