Improving Human-Robot Teaching by Quantifying and Reducing Mental Model Mismatch

TL;DR

The paper addresses misalignment between human teaching intentions and robot learning by introducing the Mental Model Mismatch (MMM) score, a feedback mechanism that leverages Large Language Models to extract teaching intents and align them with robot concepts. It defines a formal framework linking intentions to learnable concepts, computes per-iteration and cumulative mismatch scores, and uses these to generate intention-aware feedback. In a 150-participant study using a virtual Superdoku task, intention-based MMM feedback yielded higher teaching effectiveness, faster concept acquisition, and reduced misconceptions compared with performance-based or no feedback. The results demonstrate that intention-aligned feedback can enhance human-robot teaching, improve transparency of learning processes, and advance HRI interfaces for safer, more reliable collaboration.

Abstract

The rapid development of artificial intelligence and robotics has had a significant impact on our lives, with intelligent systems increasingly performing tasks traditionally performed by humans. Efficient knowledge transfer requires matching the mental model of the human teacher with the capabilities of the robot learner. This paper introduces the Mental Model Mismatch (MMM) Score, a feedback mechanism designed to quantify and reduce mismatches by aligning human teaching behavior with robot learning behavior. Using Large Language Models (LLMs), we analyze teacher intentions in natural language to generate adaptive feedback. A study with 150 participants teaching a virtual robot to solve a puzzle game shows that intention-based feedback significantly outperforms traditional performance-based feedback or no feedback. The results suggest that intention-based feedback improves instructional outcomes, improves understanding of the robot's learning process and reduces misconceptions. This research addresses a critical gap in human-robot interaction (HRI) by providing a method to quantify and mitigate discrepancies between human mental models and robot capabilities, with the goal of improving robot learning and human teaching effectiveness.

Figures (4)

• Figure 1: Interactive user interface used to teach the RiddleBot to solve the puzzle game Superdoku. The interface includes a 3x3 grid for the RiddleBot's task, an input interface for tokens, and fields for user input and feedback visualization.
• Figure 2: Box plot of scores across the three test groups. The MMM group shows a higher median score and less variability compared to the performance-based feedback and control groups. These results support Hypothesis H1, indicating that feedback incorporating the teacher's intentions significantly improves teaching effectiveness.
• Figure 3: Average number of concepts learned over time across the three test groups. The MMM group demonstrates a steeper learning curve, indicating a higher rate of concept acquisition compared to the performance-based feedback and control groups. For example, participants in the MMM group successfully taught concepts such as "unique colors" and "different shapes" earlier than other groups, demonstrating the benefits of intention-aligned feedback.
• Figure 4: Percentage of participants who thought a concept after receiving positive feedback, across multiple iterations. The MMM group consistently shows higher percentages compared to the performance-based feedback group, further supporting Hypothesis H2.