Exploring Emotion-Sensitive LLM-Based Conversational AI

TL;DR

The study investigates whether embedding emotional sensitivity in LLM-based IT support chatbots affects user perceptions of competence and satisfaction. Using a between-subjects design with 30 participants, the emotion-sensitive system guided by VADER-detected user emotion improved trust and perceived capability, while actual issue resolution remained unchanged. Manipulation checks confirmed the emotional bot more closely reflected user affect, and PANAS results showed reductions in negative emotion regardless of condition. The work highlights practical benefits for user experience in customer service AI and discusses ethical considerations around trust calibration and deployment in real-world settings.

Abstract

Conversational AI chatbots have become increasingly common within the customer service industry. Despite improvements in their emotional development, they often lack the authenticity of real customer service interactions or the competence of service providers. By comparing emotion-sensitive and emotion-insensitive LLM-based chatbots across 30 participants, we aim to explore how emotional sensitivity in chatbots influences perceived competence and overall customer satisfaction in service interactions. Additionally, we employ sentiment analysis techniques to analyze and interpret the emotional content of user inputs. We highlight that perceptions of chatbot trustworthiness and competence were higher in the case of the emotion-sensitive chatbot, even if issue resolution rates were not affected. We discuss implications of improved user satisfaction from emotion-sensitive chatbots and potential applications in support services.

Figures (2)

• Figure 1: Emotion-sensitive and -insensitive prompt engineering logic.
• Figure 2: Heatmaps of VADER-measured sentiments of user message-bot response pairs for both emotion-insensitive (left) and emotion-sensitive (right) chatbots.