A Novel Speech Analysis and Correction Tool for Arabic-Speaking Children

TL;DR

A novel technique for speech recognition using Melspectrogram and MFCC images and shows that the ResNet18 classifier on speech-to-image converted data effectively identifies mispronunciations in Arabic speech with an accuracy of 99.015\% with Mel-Spectrogram images outperforming ResNet18 with MFCC images.

Abstract

This paper introduces a new application named ArPA for Arabic kids who have trouble with pronunciation. Our application comprises two key components: the diagnostic module and the therapeutic module. The diagnostic process involves capturing the child's speech signal, preprocessing, and analyzing it using different machine learning classifiers like K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Trees as well as deep neural network classifiers like ResNet18. The therapeutic module offers eye-catching gamified interfaces in which each correctly spoken letter earns a higher avatar level, providing positive reinforcement for the child's pronunciation improvement. Two datasets were used for experimental evaluation: one from a childcare centre and the other including Arabic alphabet pronunciation recordings. Our work uses a novel technique for speech recognition using Melspectrogram and MFCC images. The results show that the ResNet18 classifier on speech-to-image converted data effectively identifies mispronunciations in Arabic speech with an accuracy of 99.015\% with Mel-Spectrogram images outperforming ResNet18 with MFCC images.

Figures (7)

• Figure 1: ArPA Architecture
• Figure 2: Correct and incorrect Pronunciation of Letter "Raa"
• Figure 3: Spectrogram and MFCC Images
• Figure 4: Classification Results of MFCC images Using ResNet18
• Figure 5: Classification Results of Mel-Spectrogram images Using ResNet18