A Few-Shot Approach to Dysarthric Speech Intelligibility Level Classification Using Transformers
Paleti Nikhil Chowdary, Vadlapudi Sai Aravind, Gorantla V N S L Vishnu Vardhan, Menta Sai Akshay, Menta Sai Aashish, Jyothish Lal. G
TL;DR
This work tackles dysarthria detection and intelligibility level classification under limited-data conditions by adopting a few-shot learning paradigm with the Whisper-large-v2 encoder and a task-specific classification head. It leverages parameter-efficient fine-tuning (PEFT) via LoRA and INT8 training to enable efficient adaptation on the UA-Speech dataset while mitigating data leakage. Binary dysarthria detection achieved 85% accuracy with high precision and specificity on medium intelligibility data, outperforming prior results on word-level tasks; multiclass classification with word-based data reached 67% accuracy, outperforming digits/letters (58%). The study demonstrates the viability of transformer-based, data-efficient approaches for pathology detection and outlines directions for reducing data requirements and comparing additional architectures.
Abstract
Dysarthria is a speech disorder that hinders communication due to difficulties in articulating words. Detection of dysarthria is important for several reasons as it can be used to develop a treatment plan and help improve a person's quality of life and ability to communicate effectively. Much of the literature focused on improving ASR systems for dysarthric speech. The objective of the current work is to develop models that can accurately classify the presence of dysarthria and also give information about the intelligibility level using limited data by employing a few-shot approach using a transformer model. This work also aims to tackle the data leakage that is present in previous studies. Our whisper-large-v2 transformer model trained on a subset of the UASpeech dataset containing medium intelligibility level patients achieved an accuracy of 85%, precision of 0.92, recall of 0.8 F1-score of 0.85, and specificity of 0.91. Experimental results also demonstrate that the model trained using the 'words' dataset performed better compared to the model trained on the 'letters' and 'digits' dataset. Moreover, the multiclass model achieved an accuracy of 67%.