IM HERE: Interaction Model for Human Effort Based Robot Engagement
Dominykas Strazdas, Magnus Jung, Jan Marquenie, Ingo Siegert, Ayoub Al-Hamadi
TL;DR
The paper tackles the challenge of generalizing engagement modeling for human-human, human-robot, and robot-robot interactions by introducing IM HERE, an effort-based framework that treats engagement as a bilateral process centered on effort. It formalizes the interaction around four states (Engaged, Passive, Requested, Buildup) and a four-stage processing loop that converts signals to engagement indicators, estimates focus, determines states, and modulates behavior to achieve social goals, while incorporating subjective and objective perspectives and miscommunication analysis. Key contributions include formal definitions, foundational concepts, axioms, Stage-wise processing, group dynamics via focus chains and F-formations, miscommunication specification, integration with previous models, and an open-source implementation for simulation. The framework aims to enable autonomous systems to participate as active, norm-aware agents in complex social settings, with applicability to education, healthcare, and multi-party interactions, and to support real-world validation and refinement.
Abstract
The effectiveness of human-robot interaction often hinges on the ability to cultivate engagement - a dynamic process of cognitive involvement that supports meaningful exchanges. Many existing definitions and models of engagement are either too vague or lack the ability to generalize across different contexts. We introduce IM HERE, a novel framework that models engagement effectively in human-human, human-robot, and robot-robot interactions. By employing an effort-based description of bilateral relationships between entities, we provide an accurate breakdown of relationship patterns, simplifying them to focus placement and four key states. This framework captures mutual relationships, group behaviors, and actions conforming to social norms, translating them into specific directives for autonomous systems. By integrating both subjective perceptions and objective states, the model precisely identifies and describes miscommunication. The primary objective of this paper is to automate the analysis, modeling, and description of social behavior, and to determine how autonomous systems can behave in accordance with social norms for full social integration while simultaneously pursuing their own social goals.