A Comprehensive Survey on Heart Sound Analysis in the Deep Learning Era

TL;DR

This paper surveys deep-learning-based heart sound analysis from 2017 to 2022, focusing on denoising, segmentation, classification, and interpretation, and contrasts with classic ML feature-based methods. It inventories public datasets (PASCAL, PhysioNet/CinC, HSS, CirCor, CirCor DigiScope, Michigan) and published algorithms, and surveys DL architectures (CNNs, RNNs, CRNNs, duration-LSTM, attention) and transfer-learning approaches (ImageNet, AudioSet, PANNs). It discusses state-of-the-art results, limitations, and open issues, including explainability, dependability, privacy, and real-life deployment. It offers future directions toward personalized, trustworthy, and hardware-aware heart sound analysis, plus a public repository for reproducibility.

Abstract

Heart sound auscultation has been applied in clinical usage for early screening of cardiovascular diseases. Due to the high demand for auscultation expertise, automatic auscultation can help with auxiliary diagnosis and reduce the burden of training professional clinicians. Nevertheless, there is a limit to classic machine learning's performance improvement in the era of big data. Deep learning has outperformed classic machine learning in many research fields, as it employs more complex model architectures with a stronger capability of extracting effective representations. Moreover, it has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were carried out before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning published in 2017--2022. This work introduces both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis. Our repository is publicly available at \url{https://github.com/zhaoren91/awesome-heart-sound-analysis}.

Figures (5)

• Figure 1: The PCG recording of a normal heart sound from the PhysioNet/CinC Database liu2016open. Frames in the middle with four states (i. e., S1, systole, S2, and diastole) are depicted.
• Figure 2: The framework of heart sound analysis involves denoising and segmentation, followed by training a classifier to produce the predictions and interpretations for clinicians and patients. The ✓ indicates a normal prediction, while the ✗ is an abnormal one. The dashes '- -' denote optional procedures.
• Figure 3: Categorisation of methods for heart sound analysis. Bold texts are DL approaches.
• Figure 4: Statistics of the literature using discriminative ML models for heart sound classification during 2017--2022. FNN: feed-forward neural network, KNN: $k$-nearest neighbour, LDC: linear discriminant classifier.
• Figure 5: Pipeline of DL models working on heart sounds. "1" indicates transfer learning; "2" illustrates deep learning on the time-frequency representation; "3" depicts end-to-end learning. The three branches can either operate in parallel or be assembled at the feature or decision level. Additionally, DL can be utilised for processing features other than raw audio signals and time-frequency representations.