Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interruption Handling for Conversational Robots

Shiye Cao, Jiwon Moon, Amama Mahmood, Victor Nikhil Antony, Ziang Xiao, Anqi Liu, Chien-Ming Huang

TL;DR

This work addresses the real-time handling of user-initiated interruptions in conversational robots by introducing an intent-aware framework that classifies interruptions into four categories: cooperative agreement, cooperative assistance, cooperative clarification, and disruptive interruption. Grounded in human-human interaction data, the system uses an LLM-based intention classifier to drive tailored interruption-handling strategies, and is implemented inside an LLM-powered social robot. Evaluation across timed decision-making and contentious discussion tasks with 21 participants shows high performance: 93.69% of interruptions are successfully handled and 88.78% of interruption intents are correctly classified, with speech recognition errors identified as the main failure source. The findings demonstrate the practical value of intention-aware interruption management for naturalistic HRI, while revealing limitations related to recognition accuracy and the need for multimodal inputs and longer-term studies.

Abstract

Interruptions, a fundamental component of human communication, can enhance the dynamism and effectiveness of conversations, but only when effectively managed by all parties involved. Despite advancements in robotic systems, state-of-the-art systems still have limited capabilities in handling user-initiated interruptions in real-time. Prior research has primarily focused on post hoc analysis of interruptions. To address this gap, we present a system that detects user-initiated interruptions and manages them in real-time based on the interrupter's intent (i.e., cooperative agreement, cooperative assistance, cooperative clarification, or disruptive interruption). The system was designed based on interaction patterns identified from human-human interaction data. We integrated our system into an LLM-powered social robot and validated its effectiveness through a timed decision-making task and a contentious discussion task with 21 participants. Our system successfully handled 93.69% (n=104/111) of user-initiated interruptions. We discuss our learnings and their implications for designing interruption-handling behaviors in conversational robots.

Interruption Handling for Conversational Robots

TL;DR

This work addresses the real-time handling of user-initiated interruptions in conversational robots by introducing an intent-aware framework that classifies interruptions into four categories: cooperative agreement, cooperative assistance, cooperative clarification, and disruptive interruption. Grounded in human-human interaction data, the system uses an LLM-based intention classifier to drive tailored interruption-handling strategies, and is implemented inside an LLM-powered social robot. Evaluation across timed decision-making and contentious discussion tasks with 21 participants shows high performance: 93.69% of interruptions are successfully handled and 88.78% of interruption intents are correctly classified, with speech recognition errors identified as the main failure source. The findings demonstrate the practical value of intention-aware interruption management for naturalistic HRI, while revealing limitations related to recognition accuracy and the need for multimodal inputs and longer-term studies.

Abstract

Interruptions, a fundamental component of human communication, can enhance the dynamism and effectiveness of conversations, but only when effectively managed by all parties involved. Despite advancements in robotic systems, state-of-the-art systems still have limited capabilities in handling user-initiated interruptions in real-time. Prior research has primarily focused on post hoc analysis of interruptions. To address this gap, we present a system that detects user-initiated interruptions and manages them in real-time based on the interrupter's intent (i.e., cooperative agreement, cooperative assistance, cooperative clarification, or disruptive interruption). The system was designed based on interaction patterns identified from human-human interaction data. We integrated our system into an LLM-powered social robot and validated its effectiveness through a timed decision-making task and a contentious discussion task with 21 participants. Our system successfully handled 93.69% (n=104/111) of user-initiated interruptions. We discuss our learnings and their implications for designing interruption-handling behaviors in conversational robots.
Paper Structure (32 sections, 3 figures, 2 tables)

This paper contains 32 sections, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Interruptions in Human-Human Interaction
  4. Interruptions in Human-Agent Interaction
  5. Interruption Handling in Human Interaction
  6. Human Conversational Data
  7. Analysis
  8. Interaction Patterns
  9. Cooperative: agreement $\rightarrow$ hold floor
  10. Cooperative: assistance $\rightarrow$ hold floor
  11. Cooperative: clarification $\rightarrow$ hold floor or yield
  12. Disruptive$\rightarrow$ hold floor or yield
  13. Interruption Handling System for Conversational Robots
  14. User-Initiated Interruptions Detection
  15. Interrupter Intention Classification
  16. ...and 17 more sections

Figures (3)

  • Figure 1: Interaction patterns of interruption handling in human conversations. The highlighted strategies were implemented in our system (Fig. \ref{['fig:interruption-handling']}). "Ack" denotes acknowledge and "Cont." denotes continue.
  • Figure 2: Metro-map-inspired diagram of the interruption handling system, illustrating how user speech flows through the interruption detection, intent classification, and interruption handling modules. It demonstrates how the system selects the handling strategy based on the predicted user intention given overlapping speech between the user and the robot. We use "Line $<$color$>$" to refer to different interruption handling paths in the figure. "Ack" denotes acknowledge and "Cont." denotes continue.
  • Figure 3: Example conversation where the system handles various types of compounded interruptions. Overlap is an example of a robot-initiated interruption. We used the Jeffersonian transcription system park2022benefits. "R" denotes robot and "P" denotes participant.