Tailored Emotional LLM-Supporter: Enhancing Cultural Sensitivity

TL;DR

This work introduces CultureCare, the first multi-cultural dataset for evaluating culturally sensitive peer emotional support, spanning four cultures with fine-grained annotations of distress, cultural signals, and support strategies. It proposes four prompting adaptation strategies for three open-source LLMs and evaluates them using automatic metrics, LLM-as-Judge, and in-culture human evaluators, including clinical psychologists. Results show that combining culture-informed role-playing with explicit cultural signals and guidelines (+cga) yields the strongest cultural awareness, while simple culture cues alone are insufficient. The study demonstrates the potential of culturally adapted LLMs for training psychology students in cross-cultural therapy, highlighting both practical utility and important safety considerations for real-world deployment.

Abstract

Large language models (LLMs) show promise in offering emotional support and generating empathetic responses for individuals in distress, but their ability to deliver culturally sensitive support remains underexplored due to a lack of resources. In this work, we introduce CultureCare, the first dataset designed for this task, spanning four cultures and including 1729 distress messages, 1523 cultural signals, and 1041 support strategies with fine-grained emotional and cultural annotations. Leveraging CultureCare, we (i) develop and test four adaptation strategies for guiding three state-of-the-art LLMs toward culturally sensitive responses; (ii) conduct comprehensive evaluations using LLM-as-a-Judge, in-culture human annotators, and clinical psychologists; (iii) show that adapted LLMs outperform anonymous online peer responses, and that simple cultural role-play is insufficient for cultural sensitivity; and (iv) explore the application of LLMs in clinical training, where experts highlight their potential in fostering cultural competence in novice therapists.

Figures (24)

• Figure 1: CultureCare: 1. The "data" panel shows a Reddit post span-annotated for emotional distress and cultural signals. Every post is paired with its top Reddit response, span-annotated for emotional support messages and cultural signals. 2. The "eval" panel shows the responses to the post returned by LLMs, with and without cultural adaptation, respectively.
• Figure 2: The CultureCare dataset construction pipeline: (1) we collect data by querying selected subreddits for candidate post-response pairs; (2) we apply rule-based and LLM-based filters to remove noisy instances (§\ref{['subsec:filter']}), e.g., that do not contain cultural signals; (3) in-culture annotators mark spans, in both posts and responses, with emotional distress, cultural signals, and support strategies; finally, a second group of annotators verify the quality of these labels and make corrections when needed.
• Figure 3: Adapted LLM responses are safe, culturally aware, and suitable for training clinical psychologists in cross-cultural therapy. L: LLM response, H: Human response.
• Figure 4: Instructions for the annotation guidelines.
• Figure 5: Prompt for extracting demographic information from CultureCare.