Leveraging Large Language Models through Natural Language Processing to provide interpretable Machine Learning predictions of mental deterioration in real time

TL;DR

This paper tackles the challenge of real-time, interpretable cognitive decline prediction in dementia by integrating a large language model (LLM)-driven conversational agent with stream-based machine learning and an explainability dashboard. A ChatGPT-3.5-turbo-based data extractor gathers 110 linguistic-emotional-conversational features from spontaneous speech, which are processed in a streaming pipeline using feature engineering, correlation and variance-based selection, and ARFC for real-time classification. The authors report robust performance, with ARFC achieving strong accuracy and recall, and provide an explainability dashboard that conveys both prediction confidence and the rationale via visual and natural-language descriptions. They also discuss ethical considerations, potential biases, and future directions such as RLHF, federated learning, and clinical deployment to broaden access and fairness in AI-assisted dementia care.

Abstract

Based on official estimates, 50 million people worldwide are affected by dementia, and this number increases by 10 million new patients every year. Without a cure, clinical prognostication and early intervention represent the most effective ways to delay its progression. To this end, Artificial Intelligence and computational linguistics can be exploited for natural language analysis, personalized assessment, monitoring, and treatment. However, traditional approaches need more semantic knowledge management and explicability capabilities. Moreover, using Large Language Models (LLMs) for cognitive decline diagnosis is still scarce, even though these models represent the most advanced way for clinical-patient communication using intelligent systems. Consequently, we leverage an LLM using the latest Natural Language Processing (NLP) techniques in a chatbot solution to provide interpretable Machine Learning prediction of cognitive decline in real-time. Linguistic-conceptual features are exploited for appropriate natural language analysis. Through explainability, we aim to fight potential biases of the models and improve their potential to help clinical workers in their diagnosis decisions. More in detail, the proposed pipeline is composed of (i) data extraction employing NLP-based prompt engineering; (ii) stream-based data processing including feature engineering, analysis, and selection; (iii) real-time classification; and (iv) the explainability dashboard to provide visual and natural language descriptions of the prediction outcome. Classification results exceed 80 % in all evaluation metrics, with a recall value for the mental deterioration class about 85 %. To sum up, we contribute with an affordable, flexible, non-invasive, personalized diagnostic system to this work.

Figures (4)

• Figure 1: System scheme.
• Figure 2: Distribution of interactions and number of words.
• Figure 3: Distribution of conversations by user.
• Figure 4: Explainability dashboard.