Exploiting ChatGPT for Diagnosing Autism-Associated Language Disorders and Identifying Distinct Features

TL;DR

This study investigates diagnosing autism-associated language disorders by leveraging ChatGPT in conjunction with ADOS-2 Module 4 data, Google's role-based speaker diarization, and structured prompts. The approach achieves higher sensitivity and PPV than conventional NLP baselines in a zero-shot setting and demonstrates robust multi-label detection of ten ASD-language features across varied social-scene prompts. Correlation and distribution analyses reveal key inter-feature relationships and scenario-specific patterns, informing targeted assessments and personalized interventions. While results are promising for AI-assisted ASD diagnostics, limitations include a small, largely homogeneous dataset and a need for multimodal data and diverse populations to enhance generalizability and clinical adoption.

Abstract

Diagnosing language disorders associated with autism is a complex challenge, often hampered by the subjective nature and variability of traditional assessment methods. Traditional diagnostic methods not only require intensive human effort but also often result in delayed interventions due to their lack of speed and precision. In this study, we explored the application of ChatGPT, a large language model, to overcome these obstacles by enhancing sensitivity and profiling linguistic features for autism diagnosis. This research utilizes ChatGPT natural language processing capabilities to simplify and improve the diagnostic process, focusing on identifying autism related language patterns. Specifically, we compared ChatGPT performance with that of conventional supervised learning models, including BERT, a model acclaimed for its effectiveness in various natural language processing tasks. We showed that ChatGPT substantially outperformed these models, achieving over 10% improvement in both sensitivity and positive predictive value, in a zero shot learning configuration. The findings underscore the model potential as a diagnostic tool, combining accuracy and applicability. We identified ten key features of autism associated language disorders across scenarios. Features such as echolalia, pronoun reversal, and atypical language usage play a critical role in diagnosing ASD and informing tailored treatment plans. Together, our findings advocate for adopting sophisticated AI tools like ChatGPT in clinical settings to assess and diagnose developmental disorders. Our approach promises enhanced diagnostic precision and supports personalized medicine, potentially transforming the evaluation landscape for autism and similar neurological conditions.

Figures (5)

• Figure 1: Correlation Coefficients Between Features of Language Deficit
• Figure 2: Correlation matrix of linguistic features $F_{1}$ to $F_{10}$ in the $S_3$ (i.e., 'Description of a Picture') scenario. The matrix shows strong correlations between features, underscoring the interdependencies that influence how visual information is described.
• Figure 3: Correlation matrix of linguistic features $F_{1}$ to $F_{10}$ in the $S_9$ (i.e., 'Cartoons') scenario. This matrix highlights correlations that elucidate the cognitive and perceptual challenges in interpreting cartoons, essential for understanding narrative contexts and humor in ASD.
• Figure 4: Detected SLD Feature Count Comparison between GPT-3.5 and GPT-4o
• Figure 5: Framework for Diagnosing Autism and Identifying Language Disorders. A4 Score: Stereotyped/Idiosyncratic Use of Words or Phrases (see Table \ref{['table:A4Score']}).