MentalAgora: A Gateway to Advanced Personalized Care in Mental Health through Multi-Agent Debating and Attribute Control

TL;DR

MentalAgora tackles the need for personalized digital mental health support by using a multi-agent debating framework among LLMs to produce tailored counseling. The method introduces three stages—Strategic Debating, Tailored Counselor Creation, and Response Generation—and utilizes a TherapyTalk dataset annotated with expert responses and attributes. Across automatic metrics, human judgments, and a user study, MentalAgora achieves higher expert alignment and user satisfaction than baselines, indicating robust attribute controllability and personalization. The approach holds potential for scalable, clinically grounded digital mental health interventions, with ethical considerations and plans for broader attribute coverage and real-world validation.

Abstract

As mental health issues globally escalate, there is a tremendous need for advanced digital support systems. We introduce MentalAgora, a novel framework employing large language models enhanced by interaction between multiple agents for tailored mental health support. This framework operates through three stages: strategic debating, tailored counselor creation, and response generation, enabling the dynamic customization of responses based on individual user preferences and therapeutic needs. We conduct experiments utilizing a high-quality evaluation dataset TherapyTalk crafted with mental health professionals, shwoing that MentalAgora generates expert-aligned and user preference-enhanced responses. Our evaluations, including experiments and user studies, demonstrate that MentalAgora aligns with professional standards and effectively meets user preferences, setting a new benchmark for digital mental health interventions.

Figures (9)

• Figure 1: MentalAgora framework overview. This diagram outlines the MentalAgora framework, showcasing its three stages: Strategic Debating, Tailored Counselor Creation, and Response Generation, which collectively enhance the creation of personalized therapeutic responses based on user-specific needs.
• Figure 2: Prompt example for tailored counselor creation. The green texts are placeholders for actual data.
• Figure 3: Prompt example for response generation. The green texts are placeholders for actual data.
• Figure 4: Automatic evaluation results of overall attribute control using Mean Absolute Error (MAE) which demonstrates the differences between scores from expert responses in TherapyTalk and generated responses across various LLMs and methods. MentalAgora outperforms all other configurations.
• Figure 5: Automatic evaluation results of overall attribute control across LLMs and methods using MAE which measures the difference of attribute scores between generated responses and given input scores. MentalAgora outperforms the other configurations compared.