Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners

Hung Manh Pham, Aaqib Saeed, Dong Ma

TL;DR

D-BETA tackles the challenge of cross-modal ECG-text representation learning by fusing generative masked auto-encoding with discriminative cross-modal objectives. It employs a transformer-based ECG encoder and a Flan-T5 text encoder, connected via a fusion module and trained with four losses, including a novel ETS loss and a nearest-neighbor negative sampling strategy (N3S). The approach demonstrates consistent improvements over state-of-the-art baselines across five public datasets in full fine-tuning, linear probing with limited data, and zero-shot settings, validating its potential to enhance automated cardiac diagnostics. The work provides extensive ablations, shows the benefits of an 8-layer ECG encoder and Flan-T5, and releases code and checkpoints to support reproducibility and future clinical NLP-ECG research.

Abstract

The accurate interpretation of Electrocardiogram (ECG) signals is pivotal for diagnosing cardiovascular diseases. Integrating ECG signals with accompanying textual reports further holds immense potential to enhance clinical diagnostics by combining physiological data and qualitative insights. However, this integration faces significant challenges due to inherent modality disparities and the scarcity of labeled data for robust cross-modal learning. To address these obstacles, we propose D-BETA, a novel framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture. D-BETA uniquely combines the strengths of generative with boosted discriminative capabilities to achieve robust cross-modal representations. This is accomplished through masked modality modeling, specialized loss functions, and an improved negative sampling strategy tailored for cross-modal alignment. Extensive experiments on five public datasets across diverse downstream tasks demonstrate that D-BETA significantly outperforms existing methods, achieving an average AUC improvement of 15% in linear probing with only one percent of training data and 2% in zero-shot performance without requiring training data over state-of-the-art models. These results highlight the effectiveness of D-BETA, underscoring its potential to advance automated clinical diagnostics through multi-modal representations. Our sample code and checkpoint are made available at https://github.com/manhph2211/D-BETA.

Boosting Masked ECG-Text Auto-Encoders as Discriminative Learners

TL;DR

D-BETA tackles the challenge of cross-modal ECG-text representation learning by fusing generative masked auto-encoding with discriminative cross-modal objectives. It employs a transformer-based ECG encoder and a Flan-T5 text encoder, connected via a fusion module and trained with four losses, including a novel ETS loss and a nearest-neighbor negative sampling strategy (N3S). The approach demonstrates consistent improvements over state-of-the-art baselines across five public datasets in full fine-tuning, linear probing with limited data, and zero-shot settings, validating its potential to enhance automated cardiac diagnostics. The work provides extensive ablations, shows the benefits of an 8-layer ECG encoder and Flan-T5, and releases code and checkpoints to support reproducibility and future clinical NLP-ECG research.

Abstract

The accurate interpretation of Electrocardiogram (ECG) signals is pivotal for diagnosing cardiovascular diseases. Integrating ECG signals with accompanying textual reports further holds immense potential to enhance clinical diagnostics by combining physiological data and qualitative insights. However, this integration faces significant challenges due to inherent modality disparities and the scarcity of labeled data for robust cross-modal learning. To address these obstacles, we propose D-BETA, a novel framework that pre-trains ECG and text data using a contrastive masked auto-encoder architecture. D-BETA uniquely combines the strengths of generative with boosted discriminative capabilities to achieve robust cross-modal representations. This is accomplished through masked modality modeling, specialized loss functions, and an improved negative sampling strategy tailored for cross-modal alignment. Extensive experiments on five public datasets across diverse downstream tasks demonstrate that D-BETA significantly outperforms existing methods, achieving an average AUC improvement of 15% in linear probing with only one percent of training data and 2% in zero-shot performance without requiring training data over state-of-the-art models. These results highlight the effectiveness of D-BETA, underscoring its potential to advance automated clinical diagnostics through multi-modal representations. Our sample code and checkpoint are made available at https://github.com/manhph2211/D-BETA.
Paper Structure (39 sections, 4 equations, 6 figures, 13 tables)

This paper contains 39 sections, 4 equations, 6 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. ECG Self-supervised Learning.
  4. ECG-Text Multi-modal Representation Learning.
  5. Method
  6. Multi-Modal Masked Auto-Encoders.
  7. ECG Encoder.
  8. Text Encoder.
  9. Fusion Module.
  10. Decoders and Self-Supervised Tasks.
  11. ECG-Text Discriminative Learners.
  12. ETS Loss Function.
  13. Nearest-neighbor-based Negative Sampling (N3S).
  14. Experiments
  15. Implementation Details.
  16. ...and 24 more sections

Figures (6)

  • Figure 1: Illustration of our contrastive masked ECG-language modeling technique.
  • Figure 2: Illustration of N3S in selecting negative samples within Flan-T5 space.
  • Figure 3: T-SNE visualization on the CSN test set.
  • Figure 4: Examples of ECG-text pairs in MIMIC IV ECG dataset gow2023mimic. We visualize three leads (I, II, V2) out of twelve.
  • Figure 5: WordCloud visualization on the top 30 common unique reports from the MIMIC IV ECG dataset.
  • ...and 1 more figures