Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SpikeVox: Towards Energy-Efficient Speech Therapy Framework with Spike-driven Generative Language Models

Rachmad Vidya Wicaksana Putra, Aadithyan Rajesh Nair, Muhammad Shafique

TL;DR

SpikeVox tackles the global lack of accessible speech therapy by integrating a spike-driven generative language model into a complete pipeline for detection, therapy generation, and feedback. It combines phoneme-level speech recognition, SpikeGPT-based disorder analysis, personalized exercise generation, and pronunciation feedback, all deployed via a REST API. In experiments on Libri-Dys and on Apple M4 hardware, SpikeVox achieves about 88% average confidence in disorder recognition and demonstrates substantial energy-efficiency due to linear-time processing and sparse spiking. This work points to a practical path toward offline, low-power speech therapy solutions that can scale globally.

Abstract

Speech disorders can significantly affect the patients capability to communicate, learn, and socialize. However, existing speech therapy solutions (e.g., therapist or tools) are still limited and costly, hence such solutions remain inadequate for serving millions of patients worldwide. To address this, state-of-the-art methods employ neural network (NN) algorithms to help accurately detecting speech disorders. However, these methods do not provide therapy recommendation as feedback, hence providing partial solution for patients. Moreover, these methods incur high energy consumption due to their complex and resource-intensive NN processing, hence hindering their deployments on low-power/energy platforms (e.g., smartphones). Toward this, we propose SpikeVox, a novel framework for enabling energy-efficient speech therapy solutions through spike-driven generative language model. Specifically, SpikeVox employs a speech recognition module to perform highly accurate speech-to-text conversion; leverages a spike-driven generative language model to efficiently perform pattern analysis for speech disorder detection and generates suitable exercises for therapy; provides guidance on correct pronunciation as feedback; as well as utilizes the REST API to enable seamless interaction for users. Experimental results demonstrate that SpikeVox achieves 88% confidence level on average in speech disorder recognition, while providing a complete feedback for therapy exercises. Therefore, SpikeVox provides a comprehensive framework for energy-efficient speech therapy solutions, and potentially addresses the significant global speech therapy access gap.

SpikeVox: Towards Energy-Efficient Speech Therapy Framework with Spike-driven Generative Language Models

TL;DR

SpikeVox tackles the global lack of accessible speech therapy by integrating a spike-driven generative language model into a complete pipeline for detection, therapy generation, and feedback. It combines phoneme-level speech recognition, SpikeGPT-based disorder analysis, personalized exercise generation, and pronunciation feedback, all deployed via a REST API. In experiments on Libri-Dys and on Apple M4 hardware, SpikeVox achieves about 88% average confidence in disorder recognition and demonstrates substantial energy-efficiency due to linear-time processing and sparse spiking. This work points to a practical path toward offline, low-power speech therapy solutions that can scale globally.

Abstract

Speech disorders can significantly affect the patients capability to communicate, learn, and socialize. However, existing speech therapy solutions (e.g., therapist or tools) are still limited and costly, hence such solutions remain inadequate for serving millions of patients worldwide. To address this, state-of-the-art methods employ neural network (NN) algorithms to help accurately detecting speech disorders. However, these methods do not provide therapy recommendation as feedback, hence providing partial solution for patients. Moreover, these methods incur high energy consumption due to their complex and resource-intensive NN processing, hence hindering their deployments on low-power/energy platforms (e.g., smartphones). Toward this, we propose SpikeVox, a novel framework for enabling energy-efficient speech therapy solutions through spike-driven generative language model. Specifically, SpikeVox employs a speech recognition module to perform highly accurate speech-to-text conversion; leverages a spike-driven generative language model to efficiently perform pattern analysis for speech disorder detection and generates suitable exercises for therapy; provides guidance on correct pronunciation as feedback; as well as utilizes the REST API to enable seamless interaction for users. Experimental results demonstrate that SpikeVox achieves 88% confidence level on average in speech disorder recognition, while providing a complete feedback for therapy exercises. Therefore, SpikeVox provides a comprehensive framework for energy-efficient speech therapy solutions, and potentially addresses the significant global speech therapy access gap.
Paper Structure (17 sections, 16 equations, 3 figures, 1 table)

This paper contains 17 sections, 16 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. State-of-the-art in Speech Therapy and Their Limitations
  3. Associated Research Challenges
  4. Our Novel Contributions
  5. Related Work
  6. The SpikeVox Framework
  7. Speech Recognition Module
  8. Speech Pattern Analysis
  9. Speech Therapy Generation
  10. Feedback Module
  11. Implementation using the REST API
  12. Experimental Methodology
  13. Results and Discussion
  14. Speech Analysis Performance
  15. SpikeVox Processing Flow
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Overview of our SpikeVox framework, with its key design components highlighted in blue.
  • Figure 2: Display of the SpikeVox's dashboard for (a) analysis page; (b) therapy generation page; and (c) feedback page.
  • Figure 3: Performance of SpikeVox for its confidence in (a) transcription process considering the severity level of disorders, and (b) issue categorization process.