MAMA-Memeia! Multi-Aspect Multi-Agent Collaboration for Depressive Symptoms Identification in Memes

TL;DR

The paper addresses detecting depressive symptoms in memes by introducing RESTOREx, a dataset enriched with ground-truth human and LLM-generated explanations, and a novel multi-agent, multi-aspect framework MAMA-Memeia grounded in Cognitive Analytic Therapy (CAT). The approach employs three aspect prompts to capture Depression, Emotional, and Cultural knowledge and orchestrates independent ideation, collaborative discussion, and consensus resolution among multiple LLMs to predict seven PHQ-9 depressive symptoms. Empirical results show MAMA-Memeia achieves state-of-the-art macro-F1 and weighted-F1 improvements (7.55% and 7.78%, respectively) over prior methods, highlighting the effectiveness of LLM-based explanations and coordinated multi-agent reasoning. The work advances multimodal meme analysis in mental health, offering scalable methods while emphasizing ethical considerations and the need for careful deployment in sensitive contexts.

Abstract

Over the past years, memes have evolved from being exclusively a medium of humorous exchanges to one that allows users to express a range of emotions freely and easily. With the ever-growing utilization of memes in expressing depressive sentiments, we conduct a study on identifying depressive symptoms exhibited by memes shared by users of online social media platforms. We introduce RESTOREx as a vital resource for detecting depressive symptoms in memes on social media through the Large Language Model (LLM) generated and human-annotated explanations. We introduce MAMAMemeia, a collaborative multi-agent multi-aspect discussion framework grounded in the clinical psychology method of Cognitive Analytic Therapy (CAT) Competencies. MAMAMemeia improves upon the current state-of-the-art by 7.55% in macro-F1 and is established as the new benchmark compared to over 30 methods.

Figures (5)

• Figure 1: An example of a depressive meme from RESTOREx with labeled fine-grained depressive symptoms and explanations from the LLM agent and Human
• Figure 2: Examples of meme samples from RESTORE that were re-annotated for the curation of RESTOREx
• Figure 3: Cognitive Analytic Therapy (CAT) Competencies parry_developing_2021 adapted as guidelines form the basis for Multi-Aspect Prompting. Multiple criteria combine to form each Knowledge Aspect. We design three Aspect-specific prompts -- $P_d$ (Depression Knowledge), $P_e$ (Emotional Knowledge), and $P_c$ (Cultural Knowledge) -- to generate the corresponding explanations -- $E_d$, $E_e$, and $E_c$. We use these explanations individually and concatenated as $<E_d , E_e , E_c>$ for our experiments.
• Figure 4: Overview of the MAMA-Memeia framework with Gemini-2.0-flash , Claude 3.5 Sonnet , GPT-4o , consisting of three phases- (1) Independent Ideation: Agents generate their initial predictions and reasoning, (2) Collaborative Discussion: Agents collaborate to discuss and reconsider their predictions, (3) Consensus Resolution: Weighted vote on final confidence estimates to determine predicted labels.
• Figure 5: Meme example with predicted labels from GPT-4o, Claude 3.5 Sonnet, Gemini-2.0-flash and MAMA-Memeia.