Human strategies for correcting `human-robot' errors during a laundry sorting task

TL;DR

The paper addresses how people communicate with domestic robots during failure, aiming to inform more natural and effective HRI in home tasks. Using a Wizard-of-Oz design with professional actors as Laundrobot, the authors analyze verbal and multimodal signals across three error modes during a laundry-sorting task, yielding 7729 speech instances. Key contributions include identifying patterns such as pattern repetition, desensitisation, and strategies like correcting/teaching, taking responsibility, and frustration, as well as nuanced attitudes toward anthropomorphic design. The findings offer actionable insights for designing autonomous cobots that detect human notice of errors and switch to learning modes, and they propose building a recovery-action library to guide future domestic robot interactions.

Abstract

Mental models and expectations underlying human-human interaction (HHI) inform human-robot interaction (HRI) with domestic robots. To ease collaborative home tasks by improving domestic robot speech and behaviours for human-robot communication, we designed a study to understand how people communicated when failure occurs. To identify patterns of natural communication, particularly in response to robotic failures, participants instructed Laundrobot to move laundry into baskets using natural language and gestures. Laundrobot either worked error-free, or in one of two error modes. Participants were not advised Laundrobot would be a human actor, nor given information about error modes. Video analysis from 42 participants found speech patterns, included laughter, verbal expressions, and filler words, such as ``oh'' and ``ok'', also, sequences of body movements, including touching one's own face, increased pointing with a static finger, and expressions of surprise. Common strategies deployed when errors occurred, included correcting and teaching, taking responsibility, and displays of frustration. The strength of reaction to errors diminished with exposure, possibly indicating acceptance or resignation. Some used strategies similar to those used to communicate with other technologies, such as smart assistants. An anthropomorphic robot may not be ideally suited to this kind of task. Laundrobot's appearance, morphology, voice, capabilities, and recovery strategies may have impacted how it was perceived. Some participants indicated Laundrobot's actual skills were not aligned with expectations; this made it difficult to know what to expect and how much Laundrobot understood. Expertise, personality, and cultural differences may affect responses, however these were not assessed.

Figures (2)

• Figure 1: Study setup including symbols used to designate locations. Participant shown placing an item of clothing in the position designated pick up point for Laundrobot.
• Figure 2: Example of a participant's (P) pattern repetition in response to errors committed by Laundrobot (R).