Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Foundation Model for Cardiac Time Series via Masked Latent Attention

Moritz Vandenhirtz, Samuel Ruipérez-Campillo, Simon Böhi, Sonia Laguna, Irene Cannistraci, Andrea Agostini, Ece Ozkan, Thomas M. Sutter, Julia E. Vogt

Abstract

Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent foundation models (FMs) have shown promise for learning transferable ECG representations, most existing pretraining approaches treat leads as independent channels and fail to explicitly leverage their strong structural redundancy. We introduce the latent attention masked autoencoder (LAMAE) FM that directly exploits this structure by learning cross-lead connection mechanisms during self-supervised pretraining. Our approach models higher-order interactions across leads through latent attention, enabling permutation-invariant aggregation and adaptive weighting of lead-specific representations. We provide empirical evidence on the Mimic-IV-ECG database that leveraging the cross-lead connection constitutes an effective form of structural supervision, improving representation quality and transferability. Our method shows strong performance in predicting ICD-10 codes, outperforming independent-lead masked modeling and alignment-based baselines.

Foundation Model for Cardiac Time Series via Masked Latent Attention

Abstract

Electrocardiograms (ECGs) are among the most widely available clinical signals and play a central role in cardiovascular diagnosis. While recent foundation models (FMs) have shown promise for learning transferable ECG representations, most existing pretraining approaches treat leads as independent channels and fail to explicitly leverage their strong structural redundancy. We introduce the latent attention masked autoencoder (LAMAE) FM that directly exploits this structure by learning cross-lead connection mechanisms during self-supervised pretraining. Our approach models higher-order interactions across leads through latent attention, enabling permutation-invariant aggregation and adaptive weighting of lead-specific representations. We provide empirical evidence on the Mimic-IV-ECG database that leveraging the cross-lead connection constitutes an effective form of structural supervision, improving representation quality and transferability. Our method shows strong performance in predicting ICD-10 codes, outperforming independent-lead masked modeling and alignment-based baselines.

Paper Structure

This paper contains 11 sections, 2 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Latent Attention for Multi-Lead Integration
  4. ECG-LAMAE FM
  5. Experiments and Results
  6. Conclusion
  7. Materials
  8. On ICD codes and further details on those used in this study
  9. Supplementary Results: Fine-grained Analysis
  10. Fine-grained Classification Results.
  11. Performance of LAMAE under Varying Supervision.

Figures (2)

  • Figure 1: Framework overview. (left) Each ECG is separated into $12$ leads, which are encoded separately and subsequently processed jointly through a latent attention transformer. The training objective is masked reconstruction. (right) The predictions are based on the latent attention's CLS token.
  • Figure 2: Label efficiency under finetuning. Performance curves of the macro-averaged AUROC over all $228$ Chapter IX codes, as a function of the number of training studies used for finetuning.