Theory of Mind for Explainable Human-Robot Interaction

TL;DR

The paper addresses the lack of rigorous evaluation of ToM-based explanations in HRI and argues for treating ToM as a form of XAI. It evaluates ToM studies using the VXAI framework to assess seven desiderata and finds that while Parsimony and Plausibility are commonly satisfied, Fidelity and Coverage are largely neglected. The results reveal that current ToM approaches often fail to faithfully reflect the robot's internal reasoning or to cover a broad range of interactions, limiting trust and generalizability. The authors advocate a unified ToM–XAI paradigm that prioritizes user-centered explanations with fidelity to model reasoning, outlining directions such as Bayesian RL and explainable RL to improve adaptivity and interpretability in HRI.

Abstract

Within the context of human-robot interaction (HRI), Theory of Mind (ToM) is intended to serve as a user-friendly backend to the interface of robotic systems, enabling robots to infer and respond to human mental states. When integrated into robots, ToM allows them to adapt their internal models to users' behaviors, enhancing the interpretability and predictability of their actions. Similarly, Explainable Artificial Intelligence (XAI) aims to make AI systems transparent and interpretable, allowing humans to understand and interact with them effectively. Since ToM in HRI serves related purposes, we propose to consider ToM as a form of XAI and evaluate it through the eValuation XAI (VXAI) framework and its seven desiderata. This paper identifies a critical gap in the application of ToM within HRI, as existing methods rarely assess the extent to which explanations correspond to the robot's actual internal reasoning. To address this limitation, we propose to integrate ToM within XAI frameworks. By embedding ToM principles inside XAI, we argue for a shift in perspective, as current XAI research focuses predominantly on the AI system itself and often lacks user-centered explanations. Incorporating ToM would enable a change in focus, prioritizing the user's informational needs and perspective.