Mirror Eyes: Explainable Human-Robot Interaction at a Glance

TL;DR

This work addresses ambiguity in natural language-driven human-robot collaboration by introducing Mirror Eyes, a reflection-like overlay on screen-based eyes mounted on a physically movable robot head. The authors integrate an LLM-based agent with a mobile head, a gaze control system, and an eye-reflection module to provide spatial references for attended targets, without requiring user training. In a 33-participant study comparing Eyes-Only versus Mirror Eyes, Mirror Eyes improved subjective information processing awareness, enabled earlier interruption of erroneous actions, and elevated overall user experience, suggesting intuitive, training-free benefits for cooperative HRI. The findings indicate that perceptual cues from mirrored eye reflections can enhance explainability beyond face-target scenarios and motivate further exploration of eye-based cues in mobile, multimodal robots.

Abstract

The gaze of a person tends to reflect their interest. This work explores what happens when this statement is taken literally and applied to robots. Here we present a robot system that employs a moving robot head with a screen-based eye model that can direct the robot's gaze to points in physical space and present a reflection-like mirror image of the attended region on top of each eye. We conducted a user study with 33 participants, who were asked to instruct the robot to perform pick-and-place tasks, monitor the robot's task execution, and interrupt it in case of erroneous actions. Despite a deliberate lack of instructions about the role of the eyes and a very brief system exposure, participants felt more aware about the robot's information processing, detected erroneous actions earlier, and rated the user experience higher when eye-based mirroring was enabled compared to non-reflective eyes. These results suggest a beneficial and intuitive utilization of the introduced method in cooperative human-robot interaction.

Figures (9)

• Figure 1: Robot platform with the proposed Mirror Eye-enhanced head. Here the robot's task is to put the red bottle onto the red plate. In the displayed snapshot the robot has grasped the bottle and plans to put it onto the red plate next. The Mirror Eyes indicate this plan by showing the target location prior to plan execution.
• Figure 2: Kinematics of the head-eye coordination model. Left: top view, right: 3D view including neck articulation with pan and tilt joints.
• Figure 3: Selected eye-based expressions. (Top row) Human-inspired expression primitives: 1. stylized iris and pupil, 2. reduced pupil size, 3. positive state, 4. negative state, 5. eyes closed, (Bottom row) Superhuman augmentations: 6. processing animation, 7. color coding of robot state, 8. Mirror Eyes focused on a person (blurred), 9. Mirror Eyes focused on objects at lower reflection opacity + loading animation indicating processing, 10. Mirror Eyes focus on an object with a brief overexposure (flash) indicating first registration.
• Figure 4: Exemplary action sequence and close up images of the robot head in the Mirror Eyes (solid blue frame) and Eyes-Only (dashed purple frame) conditions. Sequence description: 1. The robot looks at a person standing in front of it. The person gives the instruction "put the red bottle onto the red plate" 2. The robot looks at the bottle it intends to pick up. 3. The robot looks at the plate it intends to place the bottle on while moving the bottle. 4. The robot looks at the joint location of the bottle and the plate after completing the pick-and-place action.
• Figure 5: Experiment procedure for individual participants.