Design and Challenges of Mental Health Assessment Tools Based on Natural Language Interaction
Yixue Cai, Xiyan Su, Dongpeng Yao, Rongduo Han, Nan Gao, Haining Zhang
TL;DR
This paper investigates designing mental health assessment tools that operate through natural language interaction with large language models to address limitations of traditional self-report methods, such as subjectivity and privacy concerns. It describes the development of an interactive prototype, the Intelligent Mental Health Tool, which converts standardized instruments like the SCL-90 into conversational prompts and adapts questioning via an LLM-based dialogue system. Through semi-structured interviews with 11 mental health professionals, the study identifies design opportunities and critical challenges—privacy, algorithmic bias, cross-cultural applicability, empathy, and crisis intervention—while proposing a design methodology for adaptive questionnaires and multimodal data integration. The findings suggest that AI-driven, adaptive questioning can enhance user engagement and data authenticity, but clinical deployment requires substantial improvements in empathy, accuracy, and crisis response, underscoring the need for rigorous validation and careful integration into clinical workflows.
Abstract
Mental health assessments are of central importance to individuals' well-being. Conventional assessment methodologies predominantly depend on clinical interviews and standardised self-report questionnaires. Nevertheless, the efficacy of these methodologies is frequently impeded by factors such as subjectivity, recall bias, and accessibility issues. Furthermore, concerns regarding bias and privacy may result in misreporting in data collected through self-reporting in mental health research. The present study examined the design opportunities and challenges inherent in the development of a mental health assessment tool based on natural language interaction with large language models (LLMs). An interactive prototype system was developed using conversational AI for non-invasive mental health assessment, and was evaluated through semi-structured interviews with 11 mental health professionals (six counsellors and five psychiatrists). The analysis identified key design considerations for future development, highlighting how AI-driven adaptive questioning could potentially enhance the reliability of self-reported data while identifying critical challenges, including privacy protection, algorithmic bias, and cross-cultural applicability. This study provides an empirical foundation for mental health technology innovation by demonstrating the potential and limitations of natural language interaction in mental health assessment.