EnECG: Efficient Ensemble Learning for Electrocardiogram Multi-task Foundation Model

TL;DR

The experimental results demonstrate that by minimizing the scope of fine-tuning, EnECG can help reduce computational and memory costs while maintaining the strong representational power of foundation models.

Abstract

Electrocardiogram (ECG) analysis plays a vital role in the early detection, monitoring, and management of various cardiovascular conditions. While existing models have achieved notable success in ECG interpretation, they fail to leverage the interrelated nature of various cardiac abnormalities. Conversely, developing a specific model capable of extracting all relevant features for multiple ECG tasks remains a significant challenge. Large-scale foundation models, though powerful, are not typically pretrained on ECG data, making full re-training or fine-tuning computationally expensive. To address these challenges, we propose EnECG(Mixture of Experts-based Ensemble Learning for ECG Multi-tasks), an ensemble-based framework that integrates multiple specialized foundation models, each excelling in different aspects of ECG interpretation. Instead of relying on a single model or single task, EnECG leverages the strengths of multiple specialized models to tackle a variety of ECG-based tasks. To mitigate the high computational cost of full re-training or fine-tuning, we introduce a lightweight adaptation strategy: attaching dedicated output layers to each foundation model and applying Low-Rank Adaptation (LoRA) only to these newly added parameters. We then adopt a Mixture of Experts (MoE) mechanism to learn ensemble weights, effectively combining the complementary expertise of individual models. Our experimental results demonstrate that by minimizing the scope of fine-tuning, EnECG can help reduce computational and memory costs while maintaining the strong representational power of foundation models. This framework not only enhances feature extraction and predictive performance but also ensures practical efficiency for real-world clinical applications. The code is available at https://github.com/yuhaoxu99/EnECG.git.

Figures (3)

• Figure 1: Integrated Gradient Saliency Maps for the RR interval estimation task. As shown, MOMENT primarily focuses on the PR interval, TEMPO draws attention to both the PR interval and ST segment, and ECG-FM emphasizes the PR interval and QRS complex.
• Figure 2: The framework of EnECG. The EnECG framework comprises three main steps. ① Because each pretrained foundation model $\left(M_1, M_2, \ldots, M_N\right)$ requires a specific input length, we downsample the ECG and feed it into the frozen model. We then add a FFN and fine-tune it to obtain $\left(\hat{y}_1, \hat{y}_2, \ldots, \hat{y}_N\right)$. ② To reduce training costs, we select a subset of ECG leads and input them into the Mixture of Experts (MoE), which outputs gating probabilities $W$. ③ Finally, we ensemble the results via the weighted sum $\hat{y} = \sum_{i=1}^{N} W_i \hat{y}_i$.
• Figure 3: Training efficiency in the RR interval estimation task. We evaluate both GPU memory usage and throughput alongside model performance. In (a), models closer to the bottom-left corner exhibit better performance with lower memory consumption. In (b), models closer to the bottom-right corner demonstrate better performance and higher training efficiency.