Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A CNN-based Local-Global Self-Attention via Averaged Window Embeddings for Hierarchical ECG Analysis

Arthur Buzelin, Pedro Robles Dutenhefner, Turi Rezende, Luisa G. Porfirio, Pedro Bento, Yan Aquino, Jose Fernandes, Caio Santana, Gabriela Miana, Gisele L. Pappa, Antonio Ribeiro, Wagner Meira

TL;DR

The paper addresses the need to capture both fine-grained ECG waveform morphology and long-range rhythm patterns for accurate abnormality classification. It introduces Local-Global Attention ECG (LGA-ECG), which derives local queries from averaged overlapping window embeddings while attending to global keys and values, integrated within a CNN front-end and a cascade of LGA transformer blocks. On CODE-15, LGA-ECG achieves a new state-of-the-art F1 of 0.885 and accuracy of 0.994, with ablation studies confirming the superiority of the local-global attention over multiple baselines and configurations. The work demonstrates robust ECG abnormality detection with potential clinical impact and outlines future directions in self-supervised pretraining and domain adaptation to improve generalization across diverse populations and recording settings.

Abstract

Cardiovascular diseases remain the leading cause of global mortality, emphasizing the critical need for efficient diagnostic tools such as electrocardiograms (ECGs). Recent advancements in deep learning, particularly transformers, have revolutionized ECG analysis by capturing detailed waveform features as well as global rhythm patterns. However, traditional transformers struggle to effectively capture local morphological features that are critical for accurate ECG interpretation. We propose a novel Local-Global Attention ECG model (LGA-ECG) to address this limitation, integrating convolutional inductive biases with global self-attention mechanisms. Our approach extracts queries by averaging embeddings obtained from overlapping convolutional windows, enabling fine-grained morphological analysis, while simultaneously modeling global context through attention to keys and values derived from the entire sequence. Experiments conducted on the CODE-15 dataset demonstrate that LGA-ECG outperforms state-of-the-art models and ablation studies validate the effectiveness of the local-global attention strategy. By capturing the hierarchical temporal dependencies and morphological patterns in ECG signals, this new design showcases its potential for clinical deployment with robust automated ECG classification.

A CNN-based Local-Global Self-Attention via Averaged Window Embeddings for Hierarchical ECG Analysis

TL;DR

The paper addresses the need to capture both fine-grained ECG waveform morphology and long-range rhythm patterns for accurate abnormality classification. It introduces Local-Global Attention ECG (LGA-ECG), which derives local queries from averaged overlapping window embeddings while attending to global keys and values, integrated within a CNN front-end and a cascade of LGA transformer blocks. On CODE-15, LGA-ECG achieves a new state-of-the-art F1 of 0.885 and accuracy of 0.994, with ablation studies confirming the superiority of the local-global attention over multiple baselines and configurations. The work demonstrates robust ECG abnormality detection with potential clinical impact and outlines future directions in self-supervised pretraining and domain adaptation to improve generalization across diverse populations and recording settings.

Abstract

Cardiovascular diseases remain the leading cause of global mortality, emphasizing the critical need for efficient diagnostic tools such as electrocardiograms (ECGs). Recent advancements in deep learning, particularly transformers, have revolutionized ECG analysis by capturing detailed waveform features as well as global rhythm patterns. However, traditional transformers struggle to effectively capture local morphological features that are critical for accurate ECG interpretation. We propose a novel Local-Global Attention ECG model (LGA-ECG) to address this limitation, integrating convolutional inductive biases with global self-attention mechanisms. Our approach extracts queries by averaging embeddings obtained from overlapping convolutional windows, enabling fine-grained morphological analysis, while simultaneously modeling global context through attention to keys and values derived from the entire sequence. Experiments conducted on the CODE-15 dataset demonstrate that LGA-ECG outperforms state-of-the-art models and ablation studies validate the effectiveness of the local-global attention strategy. By capturing the hierarchical temporal dependencies and morphological patterns in ECG signals, this new design showcases its potential for clinical deployment with robust automated ECG classification.

Paper Structure

This paper contains 18 sections, 22 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Neural Networks for ECG
  4. Local Attention
  5. Methods
  6. Local-Global Self-Attention
  7. Transformer Block with Local-Global Self-Attention
  8. Overall Model Architecture
  9. Experiment setup
  10. Datasets
  11. Implementation details and Benchmarks
  12. Results
  13. Ablations
  14. Alternative Attention Mechanisms
  15. Positional Encoding
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Mean query extraction process for each ECG window.
  • Figure 2: Local-global self-attention operation for one ECG embedding window.
  • Figure 3: Overall architecture of the proposed LGA network, integrating the convolutional front-end, composed of four repeated ResBlocks (right), with transformer blocks utilizing local-global self-attention (left).
  • Figure 4: Comparison of the average Precision, Recall, and F1 Score between the proposed LGA-ECG model and human performance.
  • Figure 5: F1-score comparison across different window sizes.