Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework

Nobin Sarwar

TL;DR

This work addresses privacy concerns in deploying LLMs for mental health by framing a Federated Learning approach, FedMentalCare, that couples FL with Low-Rank Adaptation (LoRA) to fine-tune transformer models on-device. By keeping raw data local and learning compact adapters with updates $\\Delta W = BA$ such that $W = W_0 + \Delta W$ and $A \in \mathbb{R}^{r \times k}$, $B \in \mathbb{R}^{d \times r}$, the framework reduces communication and computation while maintaining regulatory compliance with HIPAA and GDPR. The authors study the impact of client data volume and evaluate small models like MobileBERT and MiniLM in FL settings, reporting results on the Dreaddit stress-detection task to demonstrate privacy-preserving viability with modest performance trade-offs. The paper also presents an ablation study showing how increasing the number of clients and local epochs can boost accuracy and F1, offering practical guidance for deploying FL-based mental health analytics at scale. Overall, FedMentalCare provides a scalable, privacy-conscious path for real-world mental health analysis using distributed data sources.

Abstract

With the increasing prevalence of mental health conditions worldwide, AI-powered chatbots and conversational agents have emerged as accessible tools to support mental health. However, deploying Large Language Models (LLMs) in mental healthcare applications raises significant privacy concerns, especially regarding regulations like HIPAA and GDPR. In this work, we propose FedMentalCare, a privacy-preserving framework that leverages Federated Learning (FL) combined with Low-Rank Adaptation (LoRA) to fine-tune LLMs for mental health analysis. We investigate the performance impact of varying client data volumes and model architectures (e.g., MobileBERT and MiniLM) in FL environments. Our framework demonstrates a scalable, privacy-aware approach for deploying LLMs in real-world mental healthcare scenarios, addressing data security and computational efficiency challenges.

FedMentalCare: Towards Privacy-Preserving Fine-Tuned LLMs to Analyze Mental Health Status Using Federated Learning Framework

TL;DR

This work addresses privacy concerns in deploying LLMs for mental health by framing a Federated Learning approach, FedMentalCare, that couples FL with Low-Rank Adaptation (LoRA) to fine-tune transformer models on-device. By keeping raw data local and learning compact adapters with updates such that and , , the framework reduces communication and computation while maintaining regulatory compliance with HIPAA and GDPR. The authors study the impact of client data volume and evaluate small models like MobileBERT and MiniLM in FL settings, reporting results on the Dreaddit stress-detection task to demonstrate privacy-preserving viability with modest performance trade-offs. The paper also presents an ablation study showing how increasing the number of clients and local epochs can boost accuracy and F1, offering practical guidance for deploying FL-based mental health analytics at scale. Overall, FedMentalCare provides a scalable, privacy-conscious path for real-world mental health analysis using distributed data sources.

Abstract

With the increasing prevalence of mental health conditions worldwide, AI-powered chatbots and conversational agents have emerged as accessible tools to support mental health. However, deploying Large Language Models (LLMs) in mental healthcare applications raises significant privacy concerns, especially regarding regulations like HIPAA and GDPR. In this work, we propose FedMentalCare, a privacy-preserving framework that leverages Federated Learning (FL) combined with Low-Rank Adaptation (LoRA) to fine-tune LLMs for mental health analysis. We investigate the performance impact of varying client data volumes and model architectures (e.g., MobileBERT and MiniLM) in FL environments. Our framework demonstrates a scalable, privacy-aware approach for deploying LLMs in real-world mental healthcare scenarios, addressing data security and computational efficiency challenges.

Paper Structure

This paper contains 16 sections, 4 equations, 3 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Backgrounds
  3. Large Language Models
  4. Federated Learning for Mental Health
  5. LLMs in Healthcare and Mental Healthcare Solutions
  6. Method
  7. Problem Statement
  8. Parameter-Efficient Fine-Tuning
  9. FedMentalCare Framework
  10. Experiment
  11. Datasets
  12. Implementation Details
  13. State-of-the-Art Performance Comparison
  14. Ablation Study
  15. Conclusion
  16. ...and 1 more sections

Figures (3)

  • Figure 1: Proposed FedMentalCare Framework: The architecture integrates Federated Learning (FL) with Low-Rank Adaptation (LoRA) to efficiently fine-tune Large Language Models (LLMs) on client devices for mental healthcare applications. The server initializes a global model with LoRA configuration, clients fine-tune the global model locally, and the server aggregates client updates via Federated Averaging (FedAvg), ensuring privacy and optimizing computational efficiency.
  • Figure 2: Word Cloud for Dreaddit Dataset.
  • Figure 3: Model Accuracy Comparison.