Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Scattering Transformer: A Training-Free Transformer Architecture for Heart Murmur Detection

Rami Zewail

TL;DR

The paper addresses the challenge of detecting heart murmurs in settings with limited labeled data and constrained computing resources. It introduces the Scattering Transformer, a training-free architecture that merges a fixed wavelet scattering front-end with a context-aware, parameter-free transformer-style module, followed by an SVM classifier. On CirCor DigiScope, it achieves a weighted accuracy of 0.786 and UAR of 0.697, reaching competitive performance relative to large pre-trained models while avoiding backpropagation. The results suggest a practical, data- and energy-efficient baseline for biomedical signal analysis that could extend to other data-scarce domains.

Abstract

In an attempt to address the need for skilled clinicians in heart sound interpretation, recent research efforts on automating cardiac auscultation have explored deep learning approaches. The majority of these approaches have been based on supervised learning that is always challenged in occasions where training data is limited. More recently, there has been a growing interest in potentials of pre-trained self-supervised audio foundation models for biomedical end tasks. Despite exhibiting promising results, these foundational models are typically computationally intensive. Within the context of automatic cardiac auscultation, this study explores a lightweight alternative to these general-purpose audio foundation models by introducing the Scattering Transformer, a novel, training-free transformer architecture for heart murmur detection. The proposed method leverages standard wavelet scattering networks by introducing contextual dependencies in a transformer-like architecture without any backpropagation. We evaluate our approach on the public CirCor DigiScope dataset, directly comparing it against leading general-purpose foundational models. The Scattering Transformer achieves a Weighted Accuracy(WAR) of 0.786 and an Unweighted Average Recall(UAR) of 0.697, demonstrating performance highly competitive with contemporary state of the art methods. This study establishes the Scattering Transformer as a viable and promising alternative in resource-constrained setups.

Scattering Transformer: A Training-Free Transformer Architecture for Heart Murmur Detection

TL;DR

The paper addresses the challenge of detecting heart murmurs in settings with limited labeled data and constrained computing resources. It introduces the Scattering Transformer, a training-free architecture that merges a fixed wavelet scattering front-end with a context-aware, parameter-free transformer-style module, followed by an SVM classifier. On CirCor DigiScope, it achieves a weighted accuracy of 0.786 and UAR of 0.697, reaching competitive performance relative to large pre-trained models while avoiding backpropagation. The results suggest a practical, data- and energy-efficient baseline for biomedical signal analysis that could extend to other data-scarce domains.

Abstract

In an attempt to address the need for skilled clinicians in heart sound interpretation, recent research efforts on automating cardiac auscultation have explored deep learning approaches. The majority of these approaches have been based on supervised learning that is always challenged in occasions where training data is limited. More recently, there has been a growing interest in potentials of pre-trained self-supervised audio foundation models for biomedical end tasks. Despite exhibiting promising results, these foundational models are typically computationally intensive. Within the context of automatic cardiac auscultation, this study explores a lightweight alternative to these general-purpose audio foundation models by introducing the Scattering Transformer, a novel, training-free transformer architecture for heart murmur detection. The proposed method leverages standard wavelet scattering networks by introducing contextual dependencies in a transformer-like architecture without any backpropagation. We evaluate our approach on the public CirCor DigiScope dataset, directly comparing it against leading general-purpose foundational models. The Scattering Transformer achieves a Weighted Accuracy(WAR) of 0.786 and an Unweighted Average Recall(UAR) of 0.697, demonstrating performance highly competitive with contemporary state of the art methods. This study establishes the Scattering Transformer as a viable and promising alternative in resource-constrained setups.

Paper Structure

This paper contains 25 sections, 8 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related work
  3. The Heart Murmur Detection Research Landscape
  4. Large-Scale Pre-trained and Foundation Models
  5. Hybrid Architectures and Data-Efficient Methods
  6. Positioning of the Proposed Method
  7. Methodology
  8. Wavelet Scattering Network Front-End
  9. Training-free Contextualization
  10. Positional Encoding
  11. Parameter-Free Scaled Dot-Product Attention
  12. Feed-Forward Network (FFN)
  13. Operational Modes
  14. Supervised Classification Head
  15. Experiments and Results
  16. ...and 10 more sections

Figures (2)

  • Figure 1: The two operational modes of the proposed Scattering Transformer architecture. Both modes utilize the same core parameter-free transformer block but differ in how the input sequence is constructed.
  • Figure 2: Ablation study comparing the performance of the proposed Scattering Transformer against the Standard Wavelet Scattering Networks (WSN) baseline. The chart displays the two primary evaluation metrics: Weighted Accuracy (W.acc) and Unweighted Average Recall (UAR).