Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Robust Cross-Etiology and Speaker-Independent Dysarthric Speech Recognition

Satwinder Singh, Qianli Wang, Zihan Zhong, Clarion Mendes, Mark Hasegawa-Johnson, Waleed Abdulla, Seyed Reza Shahamiri

TL;DR

The study tackles speaker-independent dysarthric speech recognition by leveraging SAP-1005 and TORGO to assess cross-speaker and cross-etiology generalization. It adopts a medium-sized multilingual Whisper model, employing 30-second chunking to mitigate hallucinations and fine-tuning on SAP-1005, achieving a CER of 6.99% and a WER of 10.71% on SAP-1005. In cross-etiology transfer, the model achieves a CER of 25.08% and a WER of 39.56% on TORGO, indicating transferable but more challenging generalization across etiologies. The results demonstrate robust speaker-independent performance on a large PD dysarthria dataset and suggest that cross-etiology transfer is feasible, guiding future work toward more diverse training data and improved robustness for spontaneous speech and severe impairment.

Abstract

In this paper, we present a speaker-independent dysarthric speech recognition system, with a focus on evaluating the recently released Speech Accessibility Project (SAP-1005) dataset, which includes speech data from individuals with Parkinson's disease (PD). Despite the growing body of research in dysarthric speech recognition, many existing systems are speaker-dependent and adaptive, limiting their generalizability across different speakers and etiologies. Our primary objective is to develop a robust speaker-independent model capable of accurately recognizing dysarthric speech, irrespective of the speaker. Additionally, as a secondary objective, we aim to test the cross-etiology performance of our model by evaluating it on the TORGO dataset, which contains speech samples from individuals with cerebral palsy (CP) and amyotrophic lateral sclerosis (ALS). By leveraging the Whisper model, our speaker-independent system achieved a CER of 6.99% and a WER of 10.71% on the SAP-1005 dataset. Further, in cross-etiology settings, we achieved a CER of 25.08% and a WER of 39.56% on the TORGO dataset. These results highlight the potential of our approach to generalize across unseen speakers and different etiologies of dysarthria.

Robust Cross-Etiology and Speaker-Independent Dysarthric Speech Recognition

TL;DR

The study tackles speaker-independent dysarthric speech recognition by leveraging SAP-1005 and TORGO to assess cross-speaker and cross-etiology generalization. It adopts a medium-sized multilingual Whisper model, employing 30-second chunking to mitigate hallucinations and fine-tuning on SAP-1005, achieving a CER of 6.99% and a WER of 10.71% on SAP-1005. In cross-etiology transfer, the model achieves a CER of 25.08% and a WER of 39.56% on TORGO, indicating transferable but more challenging generalization across etiologies. The results demonstrate robust speaker-independent performance on a large PD dysarthria dataset and suggest that cross-etiology transfer is feasible, guiding future work toward more diverse training data and improved robustness for spontaneous speech and severe impairment.

Abstract

In this paper, we present a speaker-independent dysarthric speech recognition system, with a focus on evaluating the recently released Speech Accessibility Project (SAP-1005) dataset, which includes speech data from individuals with Parkinson's disease (PD). Despite the growing body of research in dysarthric speech recognition, many existing systems are speaker-dependent and adaptive, limiting their generalizability across different speakers and etiologies. Our primary objective is to develop a robust speaker-independent model capable of accurately recognizing dysarthric speech, irrespective of the speaker. Additionally, as a secondary objective, we aim to test the cross-etiology performance of our model by evaluating it on the TORGO dataset, which contains speech samples from individuals with cerebral palsy (CP) and amyotrophic lateral sclerosis (ALS). By leveraging the Whisper model, our speaker-independent system achieved a CER of 6.99% and a WER of 10.71% on the SAP-1005 dataset. Further, in cross-etiology settings, we achieved a CER of 25.08% and a WER of 39.56% on the TORGO dataset. These results highlight the potential of our approach to generalize across unseen speakers and different etiologies of dysarthria.
Paper Structure (14 sections, 3 equations, 2 figures, 2 tables)

This paper contains 14 sections, 3 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data
  4. Speech Accessibility Project
  5. TORGO
  6. Our Approach
  7. The Whisper Model
  8. Methodology
  9. Results and Discussions
  10. Speaker-independent Results on SAP-1005
  11. Severity-based Results
  12. Category-based Results
  13. Cross-Etiology Results on TORGO
  14. Conclusions

Figures (2)

  • Figure 1: Strip plot showing error rate of each utterance in dev_unshared set of SAP-1005 across different sentence category and severity level.
  • Figure 2: Experimental results across different severity levels and sentence categories in the SAP-1005.