REFLEX Dataset: A Multimodal Dataset of Human Reactions to Robot Failures and Explanations
Parag Khanna, Andreas Naoum, Elmira Yadollahi, Mårten Björkman, Christian Smith
TL;DR
REFLEX addresses the need to understand human responses to both robot failures and the explanations robots provide, especially across repeated, long-term human-robot interactions. It introduces a comprehensive multimodal dataset collected from 55 participants across five explanation strategies, capturing audio, video, facial expressions, gaze, head, and body data, with per-frame annotations of failure actions, explanation levels, and interaction phases, including the emotion-likelihood metric $L$ for 48 emotions. The dataset is supported by visualization tools (Zenodo and Rerun) and enables analysis of trust, comprehension, and adaptation of explanations in HRI. This resource enables the design of more robust, adaptive, and socially intelligent robot collaborators that maintain positive engagement under failures and explanations.
Abstract
This work presents REFLEX: Robotic Explanations to FaiLures and Human EXpressions, a comprehensive multimodal dataset capturing human reactions to robot failures and subsequent explanations in collaborative settings. It aims to facilitate research into human-robot interaction dynamics, addressing the need to study reactions to both initial failures and explanations, as well as the evolution of these reactions in long-term interactions. By providing rich, annotated data on human responses to different types of failures, explanation levels, and explanation varying strategies, the dataset contributes to the development of more robust, adaptive, and satisfying robotic systems capable of maintaining positive relationships with human collaborators, even during challenges like repeated failures.