Multilingual Dysarthric Speech Assessment Using Universal Phone Recognition and Language-Specific Phonemic Contrast Modeling

TL;DR

This paper addresses scalable, cross-language intelligibility assessment for dysarthria by integrating a language-universal phoneme transcription with language-specific interpretation. The authors introduce three metrics—$PER$, $PFER$, and $PhonCov$—computed via a universal phone recognizer (UPR) and a contrast-aware mapping/alignment framework. They leverage PanPhon-based contrastive features and a weighted distance to map IPA sequences to language phoneme inventories and to align reference and predicted sequences, reporting improved correlations with clinician intelligibility scores across English, Spanish, Italian, and Tamil. The findings demonstrate metric-specific gains: $PER$ benefits from mapping+alignment, $PFER$ from alignment, and $PhonCov$ from mapping, with phoneme-level analyses linking severity to increases in deletions and feature-distances. The work offers scalable, interpretable tools for multilingual dysarthria assessment, while noting limitations such as absence of prosody and the need for broader-language validation.

Abstract

The growing prevalence of neurological disorders associated with dysarthria motivates the need for automated intelligibility assessment methods that are applicalbe across languages. However, most existing approaches are either limited to a single language or fail to capture language-specific factors shaping intelligibility. We present a multilingual phoneme-production assessment framework that integrates universal phone recognition with language-specific phoneme interpretation using contrastive phonological feature distances for phone-to-phoneme mapping and sequence alignment. The framework yields three metrics: phoneme error rate (PER), phonological feature error rate (PFER), and a newly proposed alignment-free measure, phoneme coverage (PhonCov). Analysis on English, Spanish, Italian, and Tamil show that PER benefits from the combination of mapping and alignment, PFER from alignment alone, and PhonCov from mapping. Further analyses demonstrate that the proposed framework captures clinically meaningful patterns of intelligibility degradation consistent with established observations of dysarthric speech.

Figures (4)

• Figure 1: Conceptual framework for multilingual dysarthric speech assessment proposed in yeo2025applications. The universal speech model encodes language-universal speech manifestations, while the language-specific intelligibility assessment model interprets the encoded representations based on the structure of each language.
• Figure 2: Example of automated pronunciation assessment using ASR and UPR. Symbolic outputs of UPR reveal segmental errors that directly reflect articulatory impairment, such as [S]$\rightarrow$[s] or [k]$\rightarrow$[t].
• Figure 3: Overview of the framework. (1) Universal phone recognition generates language-independent phone sequences. (2) Language-specific adaptation maps phones to phonemes, aligns predictions with references, and computes feature distances. (3) Three phoneme production metrics, including Phoneme Coverage (PhonCov), Phoneme Error Rate (PER), and Phonological Feature Error Rate (PFER), are computed and used to evaluate the framework against clinician-rated intelligibility scores.
• Figure 4: Distribution of phoneme feature error distances of consonants across severity levels. Violin plots show the density of PFER, with overlaid mean and median values. S denotes severity, and the following number indicates the severity level.