MindChat: A Privacy-preserving Large Language Model for Mental Health Support

TL;DR

MindChat introduces a privacy-preserving large language model for mental health support, underpinned by MindCorpus, a 5.7k-session synthetic counseling dataset generated through a dual-loop, six-agent framework. The approach combines federated learning with LoRA-based fine-tuning and client-level differential privacy to protect sensitive data during distributed training. Empirical results show MindChat achieves competitive automatic performance and strong privacy protection against membership inference attacks, with data diversity found to be more impactful than sheer data volume. The work advances practical deployment of private, scalable mental health LLMs and outlines directions for multi-modal extensions and ongoing privacy safeguards.

Abstract

Large language models (LLMs) have shown promise for mental health support, yet training such models is constrained by the scarcity and sensitivity of real counseling dialogues. In this article, we present MindChat, a privacy-preserving LLM for mental health support, together with MindCorpus, a synthetic multi-turn counseling dataset constructed via a multi-agent role-playing framework. To synthesize high-quality counseling data, the developed dialogue-construction framework employs a dual closed-loop feedback design to integrate psychological expertise and counseling techniques through role-playing: (i) turn-level critique-and-revision to improve coherence and counseling appropriateness within a session, and (ii) session-level strategy refinement to progressively enrich counselor behaviors across sessions. To mitigate privacy risks under decentralized data ownership, we fine-tune the base model using federated learning with parameter-efficient LoRA adapters and incorporate differentially private optimization to reduce membership and memorization risks. Experiments on synthetic-data quality assessment and counseling capability evaluation show that MindCorpus improves training effectiveness and that MindChat is competitive with existing general and counseling-oriented LLM baselines under both automatic LLM-judge and human evaluation protocols, while exhibiting reduced privacy leakage under membership inference attacks.

Figures (14)

• Figure 1: The personal situation of seekers.
• Figure 2: The proposed multi-agent cooperation architecture.
• Figure 3: The parameter-efficient privacy-preserving fine-tuning framework.
• Figure 4: Data distribution of MindCorpus.
• Figure 5: The impact of multi-agent collaboration on the quality of synthetic data.