Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MELEP: A Novel Predictive Measure of Transferability in Multi-Label ECG Diagnosis

Cuong V. Nguyen, Hieu Minh Duong, Cuong D. Do

TL;DR

MELEP introduces a novel transferability metric for multi-label ECG diagnosis that estimates downstream fine-tuning performance from a single forward pass, extending the LEEP concept to multi-label ECG tasks. By computing dummy target predictions, empirical joint and conditional distributions, and an Empirical Predictor-based likelihood, MELEP yields a transferability score $\Phi(\Theta, \mathcal{D})$ that declines with better transferability. Extensive experiments across CNN and Bi-LSTM architectures on four 12-lead ECG datasets (CSN, PTB-XL, CPSC2018, Georgia) show strong negative correlations (often $|r|>0.6$) between MELEP and actual average F1, enabling effective checkpoint selection for transfer learning with limited annotated data. The work highlights MELEP’s practicality, dataset-agnostic applicability, and potential extensions to other learning paradigms such as continual or federated ECG analysis.

Abstract

In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for Muti-label Expected Log of Empirical Predictions, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.

MELEP: A Novel Predictive Measure of Transferability in Multi-Label ECG Diagnosis

TL;DR

MELEP introduces a novel transferability metric for multi-label ECG diagnosis that estimates downstream fine-tuning performance from a single forward pass, extending the LEEP concept to multi-label ECG tasks. By computing dummy target predictions, empirical joint and conditional distributions, and an Empirical Predictor-based likelihood, MELEP yields a transferability score that declines with better transferability. Extensive experiments across CNN and Bi-LSTM architectures on four 12-lead ECG datasets (CSN, PTB-XL, CPSC2018, Georgia) show strong negative correlations (often ) between MELEP and actual average F1, enabling effective checkpoint selection for transfer learning with limited annotated data. The work highlights MELEP’s practicality, dataset-agnostic applicability, and potential extensions to other learning paradigms such as continual or federated ECG analysis.

Abstract

In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for Muti-label Expected Log of Empirical Predictions, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.
Paper Structure (12 sections, 4 equations, 6 figures, 3 tables)

This paper contains 12 sections, 4 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Materials & Methods
  3. Multi-Label Expected Log of Empirical Predictions (MELEP)
  4. Datasets
  5. Deep Learning Models
  6. Performance metric
  7. Experiments & Results
  8. Relation between MELEP and Model Performance of CNN fine-tuned on CSN
  9. Relation between MELEP and Model Performance of RNN fine-tuned on CSN
  10. Relation between MELEP and Performance of Models fine-tuned on PTB-XL
  11. MELEP for Checkpoint Selection
  12. Discussions & Conclusion

Figures (6)

  • Figure 1: ResNet1d101 Architecture
  • Figure 2: Bi-LSTM Architecture
  • Figure 3: Relation of MELEP (partitioned as four distance levels) and fine-tuning performance of ResNet1d101 on target tasks sampled from the CSN dataset. The lower the MELEP (the closer the distance), the better transferability.
  • Figure 4: Relation of MELEP (partitioned as four distance levels) and fine-tuning performance of Bi-LSTM on target tasks sampled from the CSN dataset. The lower the MELEP (the closer the distance), the better transferability.
  • Figure 5: Relation of MELEP (partitioned as four distance levels) and fine-tuning performance of ResNet1d101 and Bi-LSTM on target tasks sampled from the PTB-XL dataset.
  • ...and 1 more figures