MoPHES:Leveraging on-device LLMs as Agent for Mobile Psychological Health Evaluation and Support

TL;DR

MoPHES tackles the global shortage of accessible mental health care by delivering an on-device agent that simultaneously performs real-time mental-state prediction and engages in empathetic, multi-turn counseling, while offering professional treatment guidance. It achieves this with two on-device MiniCPM4-0.5B LLMs fine-tuned on separate domains (mental-state assessment and dialogues) and deployed via llama.cpp on Android, supported by a specialized dataset from PsyQA and EmoLLM and a custom benchmark for automatic evaluation. The work demonstrates that a compact 0.5B-parameter model can outperform larger baselines in mental health tasks, while preserving user privacy through on-device processing; it also provides a comprehensive evaluation framework covering both automated metrics and human judgments. Looking forward, the authors propose RL-based alignment and a unified single-model approach to improve efficiency and safety, with practical impact in privacy-preserving, accessible mobile mental health support. The combination of dataset-driven fine-tuning, on-device deployment, and a dedicated evaluation benchmark positions MoPHES as a versatile, privacy-friendly option for real-world digital mental health interventions.

Abstract

The 2022 World Mental Health Report calls for global mental health care reform, amid rising prevalence of issues like anxiety and depression that affect nearly one billion people worldwide. Traditional in-person therapy fails to meet this demand, and the situation is worsened by stigma. While general-purpose large language models (LLMs) offer efficiency for AI-driven mental health solutions, they underperform because they lack specialized fine-tuning. Existing LLM-based mental health chatbots can engage in empathetic conversations, but they overlook real-time user mental state assessment which is critical for professional counseling. This paper proposes MoPHES, a framework that integrates mental state evaluation, conversational support, and professional treatment recommendations. The agent developed under this framework uses two fine-tuned MiniCPM4-0.5B LLMs: one is fine-tuned on mental health conditions datasets to assess users' mental states and predict the severity of anxiety and depression; the other is fine-tuned on multi-turn dialogues to handle conversations with users. By leveraging insights into users' mental states, our agent provides more tailored support and professional treatment recommendations. Both models are also deployed directly on mobile devices to enhance user convenience and protect user privacy. Additionally, to evaluate the performance of MoPHES with other LLMs, we develop a benchmark for the automatic evaluation of mental state prediction and multi-turn counseling dialogues, which includes comprehensive evaluation metrics, datasets, and methods.

Figures (7)

• Figure 1: Overview of MoPHES. (a) We use GPT-4o-mini to label the mental conditions dataset and transform singile-turn QAs to multi-turn dialogues; (b) We fine-tune the base model MiniCPM4-0.5B on these two datasets and obtain evaluation model and dialogue model respectively, and deploy the models on mobile device to build the agent; (c) The interaction between user and agent: user can chat with agent and seek help via multi-turn conversations, and agent will support and cure user, meanwhile regular assess user's mental state and save them locally.
• Figure 2: Distribution of counseling topics.
• Figure 3: Prompt of Mental Health Conditions Labeling.
• Figure 4: Prompt of Multi-turn Dialogues Generation.
• Figure 5: Prompt of Automatic Evaluation.