Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Developing an End-to-End Framework for Predicting the Social Communication Severity Scores of Children with Autism Spectrum Disorder

Jihyun Mun, Sunhee Kim, Minhwa Chung

TL;DR

The paper addresses the challenge of objectively assessing social communication severity in children with ASD using speech data. It proposes an end-to-end framework that combines ASD-tailored automatic speech recognition (ASR) with fine-tuned pre-trained language models (PLMs) and a seed-ensemble approach, including prompt-tuning variants. The approach achieves a Pearson correlation coefficient of $r = 0.6566$ with human-rated scores, demonstrating potential, especially in data-limited settings, and shows that ASR transcripts can substitute for human transcripts in low-resource scenarios. Key contributions include an integrated ASR-PLM pipeline, a systematic comparison of traditional, manual, and prompt-based tuning strategies, and evidence that ASR-based methods can reach competitive performance when data are scarce, albeit with interpretability challenges that warrant further work.

Abstract

Autism Spectrum Disorder (ASD) is a lifelong condition that significantly influencing an individual's communication abilities and their social interactions. Early diagnosis and intervention are critical due to the profound impact of ASD's characteristic behaviors on foundational developmental stages. However, limitations of standardized diagnostic tools necessitate the development of objective and precise diagnostic methodologies. This paper proposes an end-to-end framework for automatically predicting the social communication severity of children with ASD from raw speech data. This framework incorporates an automatic speech recognition model, fine-tuned with speech data from children with ASD, followed by the application of fine-tuned pre-trained language models to generate a final prediction score. Achieving a Pearson Correlation Coefficient of 0.6566 with human-rated scores, the proposed method showcases its potential as an accessible and objective tool for the assessment of ASD.

Developing an End-to-End Framework for Predicting the Social Communication Severity Scores of Children with Autism Spectrum Disorder

TL;DR

The paper addresses the challenge of objectively assessing social communication severity in children with ASD using speech data. It proposes an end-to-end framework that combines ASD-tailored automatic speech recognition (ASR) with fine-tuned pre-trained language models (PLMs) and a seed-ensemble approach, including prompt-tuning variants. The approach achieves a Pearson correlation coefficient of with human-rated scores, demonstrating potential, especially in data-limited settings, and shows that ASR transcripts can substitute for human transcripts in low-resource scenarios. Key contributions include an integrated ASR-PLM pipeline, a systematic comparison of traditional, manual, and prompt-based tuning strategies, and evidence that ASR-based methods can reach competitive performance when data are scarce, albeit with interpretability challenges that warrant further work.

Abstract

Autism Spectrum Disorder (ASD) is a lifelong condition that significantly influencing an individual's communication abilities and their social interactions. Early diagnosis and intervention are critical due to the profound impact of ASD's characteristic behaviors on foundational developmental stages. However, limitations of standardized diagnostic tools necessitate the development of objective and precise diagnostic methodologies. This paper proposes an end-to-end framework for automatically predicting the social communication severity of children with ASD from raw speech data. This framework incorporates an automatic speech recognition model, fine-tuned with speech data from children with ASD, followed by the application of fine-tuned pre-trained language models to generate a final prediction score. Achieving a Pearson Correlation Coefficient of 0.6566 with human-rated scores, the proposed method showcases its potential as an accessible and objective tool for the assessment of ASD.
Paper Structure (17 sections, 1 figure, 2 tables)

This paper contains 17 sections, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Automatic Speech Recognition Model
  4. Fine-tuning Pre-trained Language Models
  5. Traditional Fine-tuning
  6. Manual Prompting
  7. Automated Prompting: P-Tuning
  8. Seed Ensemble for Robust Prediction
  9. Experiments
  10. Data Preparation and Dataset Description
  11. Fine-Tuning ASR Models
  12. Fine-Tuning PLMs
  13. Evaluation Metrics
  14. Results
  15. Discussion
  16. ...and 2 more sections

Figures (1)

  • Figure 1: Proposed E2E framework for automatically predicting the social communication severity scores of children with ASD