Assessing the Utility of Audio Foundation Models for Heart and Respiratory Sound Analysis

TL;DR

The paper evaluates fixed-weight audio foundation models as feature extractors for respiratory and heart sound analysis, comparing them to SOTA results obtained with fine-tuning across four tasks. Across ICBHI2017, SPRS, CirCor, and BMD-HS, general-purpose models trained on large-scale audio data largely match or approach SOTA without fine-tuning, whereas respiratory-specific models underperform in noisy conditions. Performance strongly depends on data quality: clean datasets yield strong results with fixed weights, while noisy recordings benefit from preprocessing or fine-tuning to suppress irrelevant sounds. The work provides practical insights and releases evaluation code to guide future research on foundation models for auscultation.

Abstract

Pre-trained deep learning models, known as foundation models, have become essential building blocks in machine learning domains such as natural language processing and image domains. This trend has extended to respiratory and heart sound models, which have demonstrated effectiveness as off-the-shelf feature extractors. However, their evaluation benchmarking has been limited, resulting in incompatibility with state-of-the-art (SOTA) performance, thus hindering proof of their effectiveness. This study investigates the practical effectiveness of off-the-shelf audio foundation models by comparing their performance across four respiratory and heart sound tasks with SOTA fine-tuning results. Experiments show that models struggled on two tasks with noisy data but achieved SOTA performance on the other tasks with clean data. Moreover, general-purpose audio models outperformed a respiratory sound model, highlighting their broader applicability. With gained insights and the released code, we contribute to future research on developing and leveraging foundation models for respiratory and heart sounds.

Figures (1)

• Figure 1: Network configuration for performance evaluation.