Personalizing Emotion-aware Conversational Agents? Exploring User Traits-driven Conversational Strategies for Enhanced Interaction

TL;DR

This work investigates how user traits such as gender, personality (Big Five), and culture influence interaction strategies with emotion-aware conversational agents (CAs) across five emotional contexts (neutral, happy, sad, angry, fear). Using two within-subject studies, one with a male-voiced CA and the other with a female-voiced CA, the authors combine quantitative questionnaire metrics with qualitative thematic analysis to reveal trait-based differences in how users adjust responses, engage with the CA, and propose strategic modifications. Key findings show gender as a dominant factor in emotional interaction, with women tending toward empathetic engagement and men toward pragmatic solutions; personality and culture further modulate engagement patterns and preferred emotional cues. The study provides design guidelines for trait-aware, emotionally intelligent CAs, highlighting the importance of adaptive voice, emotional alignment, and culturally sensitive interaction strategies to improve user satisfaction and interaction quality. It also points to future work in expanding demographic diversity, enabling multi-turn and multimodal emotional cues, and leveraging advanced LLMs for richer, context-aware dialogue.

Abstract

Conversational agents (CAs) are increasingly embedded in daily life, yet their ability to navigate user emotions efficiently is still evolving. This study investigates how users with varying traits -- gender, personality, and cultural background -- adapt their interaction strategies with emotion-aware CAs in specific emotional scenarios. Using an emotion-aware CA prototype expressing five distinct emotions (neutral, happy, sad, angry, and fear) through male and female voices, we examine how interaction dynamics shift across different voices and emotional contexts through empirical studies. Our findings reveal distinct variations in user engagement and conversational strategies based on individual traits, emphasizing the value of personalized, emotion-sensitive interactions. By analyzing both qualitative and quantitative data, we demonstrate that tailoring CAs to user characteristics can enhance user satisfaction and interaction quality. This work underscores the critical need for ongoing research to design CAs that not only recognize but also adaptively respond to emotional needs, ultimately supporting a diverse user groups more effectively.

Figures (13)

• Figure 1: The interfaces of our CA prototype featuring five emotive smiley icons with each representing a distinct emotion. Users are prompted to click the smiley faces, where the selected icons will turn yellow, which then initiates the conversation phase.
• Figure 2: An overview of the user study workflow.
• Figure 3: Some of the demographic information about the participants. In subfigures: Left: Study I; right: Study II.
• Figure 4: The baseline emotions of participants. Left: Study I; right: Study II.
• Figure 5: Quantitative results from Study I with male voice in CAs, showing the mean ratings across the nine metrics. Pairwise significance from post-hoc analysis is indicated as follows: $*$: $p < .05$; $***$: $p < .001$.