A Robot That Listens: Enhancing Self-Disclosure and Engagement Through Sentiment-based Backchannels and Active Listening

TL;DR

The paper addresses eliciting deep self-disclosure in human-robot conversations by designing an autonomous LLM-powered robot that combines sentiment-based backchannels with active listening. It introduces a two-component system (Sentiment-based Backchanneling and Active Listening) implemented on a Furhat robot and evaluates three conditions (Control, BC, BC+AL) with 60 participants after exclusions. Results show that the BC+AL condition increases utterances, deepens Information and Feelings in disclosures, and enhances facial engagement, indicating stronger rapport, though perceived empathy and closeness metrics did not differ across conditions. The findings support the potential of socio-emotionally intelligent listening behaviors to improve human-robot communication and inform the design of personalized robotic support systems.

Abstract

As social robots get more deeply integrated intoour everyday lives, they will be expected to engage in meaningful conversations and exhibit socio-emotionally intelligent listening behaviors when interacting with people. Active listening and backchanneling could be one way to enhance robots' communicative capabilities and enhance their effectiveness in eliciting deeper self-disclosure, providing a sense of empathy,and forming positive rapport and relationships with people.Thus, we developed an LLM-powered social robot that can exhibit contextually appropriate sentiment-based backchannelingand active listening behaviors (active listening+backchanneling) and compared its efficacy in eliciting people's self-disclosurein comparison to robots that do not exhibit any of these listening behaviors (control) and a robot that only exhibitsbackchanneling behavior (backchanneling-only). Through ourexperimental study with sixty-five participants, we found theparticipants who conversed with the active listening robot per-ceived the interactions more positively, in which they exhibited the highest self-disclosures, and reported the strongest senseof being listened to. The results of our study suggest that the implementation of active listening behaviors in social robotshas the potential to improve human-robot communication andcould further contribute to the building of deeper human-robot relationships and rapport.

Figures (4)

• Figure 1: An overview of system architecture. When a participant answers a question, the prosodic features were extracted via OpenSmile park2017backchannel and the verbal utterances were recognized via the Azure Speech SDK ASR module. The BOP module identified the appropriate timing for the robot to backchannel and the Sentiment Analyzer module classified the participant's utterance as positive, neutral, or negative to generate sentiment-based backchanneling behavior for the robot. The Active Listener module used the Azure Speech SDK ASR results to generate empathetic robot responses based on the prompted active listening principles.
• Figure 2: We compared three robot listening behaviors. In the (a) the control condition (Control), the robot only asked questions, (b) the backchanneling-only condition (BC), in which the robot produced sentiment-based backchannels while listening to the participant's response, and (c) the backchanneling+active listening behavior (BC+AL), the robot produced sentiment-based backchannels and followed up with active listening responses.
• Figure 3: We found that increasing trends of Control$<$BC$<$BC+AL in (a) the number of participants' verbal utterances, (b) the level of two types of self-disclosures (Information and Feelings) in participants' responses, and (c) the level of Engagement observed in participants' facial expressions.
• Figure 4: Participants reported statistically significant differences in their perception of the robot's Inquisitiveness, Listening, and Sense) across the three experimental conditions.