Approaching Low-Cost Cardiac Intelligence with Semi-Supervised Knowledge Distillation

TL;DR

LiteHeart tackles the practical challenge of deploying cardiac AI on wearable devices by bridging the diagnostic gap between 1-lead LCCI and 12-lead HCCI. It introduces region-aware knowledge distillation, cross-layer mutual information maximization, and semi-supervised learning to transfer rich 12-lead knowledge to compact 1-lead models, while leveraging unlabeled data. Across five multi-center datasets covering 38 CVDs, LiteHeart nearly halves the LCCI–HCCI gap and outperforms state-of-the-art KD methods, with substantial gains in macro F1, MAP, and macro AUC, plus marked improvements in inference efficiency. The approach supports scalable deployment on wearables and suggests broader applicability to other on-edge physiological signals and embodied AI tasks.

Abstract

Deploying advanced cardiac artificial intelligence for daily cardiac monitoring is hindered by its reliance on extensive medical data and high computational resources. Low-cost cardiac intelligence (LCCI) offers a promising alternative by using wearable device data, such as 1-lead electrocardiogram (ECG), but it suffers from a significant diagnostic performance gap compared to high-cost cardiac intelligence (HCCI). To bridge this gap, we propose LiteHeart, a semi-supervised knowledge distillation framework. LiteHeart introduces a region-aware distillation module to mimic how cardiologists focus on diagnostically relevant ECG regions and a cross-layer mutual information module to align the decision processes of LCCI and HCCI systems. Using a semi-supervised training strategy, LiteHeart further improves model robustness under limited supervision. Evaluated on five datasets covering over 38 cardiovascular diseases, LiteHeart substantially reduces the performance gap between LCCI and HCCI, outperforming existing methods by 4.27% to 7.10% in macro F1 score. These results demonstrate that LiteHeart significantly enhances the diagnostic capabilities of low-cost cardiac intelligence systems, paving the way for scalable, affordable, and accurate daily cardiac healthcare using wearable technologies.

Figures (15)

• Figure 1: a. High-cost cardiac intelligence utilizes 12-lead ECG signals to make CVDs diagnoses and should be deployed on high-performance computers due to its large model size. b. Low-cost cardiac intelligence utilizes 1-lead ECG signals to make CVDs diagnoses and can be deployed on mobile computing platforms because of its small model size. c. The proposed LiteHeart introduces three modules to bridge the diagnostic performance gap between low-cost and high-cost cardiac intelligence: (1) region-aware knowledge distillation; (2) cross-layer mutual information maximization; (3) semi-supervised optimization. d. Integrating the restoration net and the student net, we finally generate a robust low-cost cardiac intelligence system for daily cardiac healthcare using 1-lead ECG signals. It will first restore 12-lead ECG signals from 1-lead ECG signals and then output the corresponding CVDs diagnostic results. It can detect over 38 CVDs and show potential for daily cardiac healthcare.
• Figure 2: a. CVDs diagnostic performance of the low-cost cardiac intelligence generated by the proposed LiteHeart and the SOTA methods on different downstream datasets. The red dashed lines denote the performance of the high-cost teacher nets using 12-lead ECG signals, which serve as the performance ceiling of low-cost cardiac intelligence. The baseline of low-cost cardiac intelligence is formulated by deactivating the knowledge distillation. b. T-SNE visualization of the last layer logits output by the high-cost cardiac intelligence, the low-cost cardiac intelligence generated by LiteHeart, SOTA-1st, SOTA-2nd, and the baseline method. The circles with different colors describe the distribution of CVDs diagnoses made by different models.
• Figure 3: a. AUC of different models on various CVDs from the SMU dataset. b-c. Visualization of ROIs for diagnosing “lambda-wave” ST-elevation (STE) and premature ventricular contractions (PVC) using the Grad-GAM approach. For example, ST-elevation is difficult to diagnose using the real lead I ECG signal. With the restoration net, a restored 12-lead ECG signal can be generated using the lead I signal, and the corresponding diagnostic patterns are recovered. As shown in lead II of the restored signal, the ROIs for diagnosing “lambda-wave” ST-elevation are highlighted using the low-cost cardiac intelligence generated by LiteHeart. For comparison, we visualize the lead II of the real ECG signal and highlight the ROIs using high-cost cardiac intelligence. d. The impact of ECG lead selection on the diagnostic performance of the low-cost cardiac intelligence.
• Figure 4: a. The impact of restoration net and student net sizes on the inference efficiency of the low-cost cardiac intelligence generated by LiteHeart. b. The impact of restoration net and student net sizes on the CVDs diagnostic performance. Here, ‘Res’ and 'Stu' are the abbreviations of the restoration net and the student net, respectively. Specifically, a tiny student net has 0.26 million parameters, and a tiny restoration net has 0.36 million parameters. A small student net has 1.01 million parameters, and a small restoration net has 1.43 million parameters. A base student net has 1.60 million parameters, and a base restoration net has 5.71 million parameters.
• Figure 5: a. Ablation study of the proposed LiteHeart. The baseline of low-cost cardiac intelligence is formulated by deactivating the knowledge distillation. b. Cross-device external validation of high-cost cardiac intelligence (HCCI) and the low-cost cardiac intelligence generated by different methods. The mean value and the standard deviations of all metrics are calculated across four random seeds. c. The distribution of the diagnostic performance across all evaluated CVDs. For example, a point within the violin plot denotes the AUC of one CVD. The centroid of the points denotes the value of macro AUC for CVDs detection.