Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards objective and interpretable speech disorder assessment: a comparative analysis of CNN and transformer-based models

Malo Maisonneuve, Corinne Fredouille, Muriel Lalain, Alain Ghio, Virginie Woisard

TL;DR

The paper tackles the challenge of objective and interpretable speech disorder assessment in Head and Neck Cancer patients by evaluating a self-supervised Wav2Vec2-based phone classifier against a CNN baseline. Through multi-dataset experiments on BREF, Common Phone, and C2SI corpora, it analyzes the effects of pre-training data, model size, and fine-tuning regimes, showing that larger Wav2Vec2 models with diverse fine-tuning data achieve superior phone-level accuracy. Importantly, the study demonstrates strong correlations between phone-classification performance and expert perceptual scores, supporting the clinical relevance and interpretability potential of SSL-based representations. These findings highlight the practical impact of transformer-based speech models for robust, interpretable pathological speech analysis and set the stage for further exploration of hidden-layer interpretability to aid rehabilitation strategies.

Abstract

Head and Neck Cancers (HNC) significantly impact patients' ability to speak, affecting their quality of life. Commonly used metrics for assessing pathological speech are subjective, prompting the need for automated and unbiased evaluation methods. This study proposes a self-supervised Wav2Vec2-based model for phone classification with HNC patients, to enhance accuracy and improve the discrimination of phonetic features for subsequent interpretability purpose. The impact of pre-training datasets, model size, and fine-tuning datasets and parameters are explored. Evaluation on diverse corpora reveals the effectiveness of the Wav2Vec2 architecture, outperforming a CNN-based approach, used in previous work. Correlation with perceptual measures also affirms the model relevance for impaired speech analysis. This work paves the way for better understanding of pathological speech with interpretable approaches for clinicians, by leveraging complex self-learnt speech representations.

Towards objective and interpretable speech disorder assessment: a comparative analysis of CNN and transformer-based models

TL;DR

The paper tackles the challenge of objective and interpretable speech disorder assessment in Head and Neck Cancer patients by evaluating a self-supervised Wav2Vec2-based phone classifier against a CNN baseline. Through multi-dataset experiments on BREF, Common Phone, and C2SI corpora, it analyzes the effects of pre-training data, model size, and fine-tuning regimes, showing that larger Wav2Vec2 models with diverse fine-tuning data achieve superior phone-level accuracy. Importantly, the study demonstrates strong correlations between phone-classification performance and expert perceptual scores, supporting the clinical relevance and interpretability potential of SSL-based representations. These findings highlight the practical impact of transformer-based speech models for robust, interpretable pathological speech analysis and set the stage for further exploration of hidden-layer interpretability to aid rehabilitation strategies.

Abstract

Head and Neck Cancers (HNC) significantly impact patients' ability to speak, affecting their quality of life. Commonly used metrics for assessing pathological speech are subjective, prompting the need for automated and unbiased evaluation methods. This study proposes a self-supervised Wav2Vec2-based model for phone classification with HNC patients, to enhance accuracy and improve the discrimination of phonetic features for subsequent interpretability purpose. The impact of pre-training datasets, model size, and fine-tuning datasets and parameters are explored. Evaluation on diverse corpora reveals the effectiveness of the Wav2Vec2 architecture, outperforming a CNN-based approach, used in previous work. Correlation with perceptual measures also affirms the model relevance for impaired speech analysis. This work paves the way for better understanding of pathological speech with interpretable approaches for clinicians, by leveraging complex self-learnt speech representations.
Paper Structure (8 sections, 3 figures, 2 tables)

This paper contains 8 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Corpora
  3. Model architectures
  4. Experimental results
  5. Comparison of Wav2Vec2 models
  6. Wav2Vec2 and CNN
  7. Application to pathological speech
  8. Conclusion

Figures (3)

  • Figure 1: Confusion matrices of obstruent phones, on BREF-Int and C2SI-LEC (HC speakers only) datasets respectively.
  • Figure 2: Confusion matrices of oral/nasal phones, on BREF-Int and C2SI-LEC (HC speakers only) datasets respectively.
  • Figure 3: Scatter plots of C2SI patients and HC on perceptual evaluations by experts and balanced accuracies obtained.