AToM: Adaptive Theory-of-Mind-Based Human Motion Prediction in Long-Term Human-Robot Interactions

TL;DR

The paper tackles long-term human-robot interactions where human behavior evolves over repeated encounters. It introduces Adaptive Theory-of-Mind (AToM), a game-theoretic human internal model with parameters theta that is updated online via an Unscented Kalman Filter, eliminating recursive planning bottlenecks by solving a Nash equilibrium with an ILQSolver. The approach yields more accurate human trajectory predictions (lower ADE), safer and more efficient robot planning, and interpretable insights into how humans infer robot behavior. Validation includes simulations across multiple scenarios and a real-world user study, demonstrating both predictive performance and practical impact on planning safety and efficiency in HRI settings.

Abstract

Humans learn from observations and experiences to adjust their behaviours towards better performance. Interacting with such dynamic humans is challenging, as the robot needs to predict the humans accurately for safe and efficient operations. Long-term interactions with dynamic humans have not been extensively studied by prior works. We propose an adaptive human prediction model based on the Theory-of-Mind (ToM), a fundamental social-cognitive ability that enables humans to infer others' behaviours and intentions. We formulate the human internal belief about others using a game-theoretic model, which predicts the future motions of all agents in a navigation scenario. To estimate an evolving belief, we use an Unscented Kalman Filter to update the behavioural parameters in the human internal model. Our formulation provides unique interpretability to dynamic human behaviours by inferring how the human predicts the robot. We demonstrate through long-term experiments in both simulations and real-world settings that our prediction effectively promotes safety and efficiency in downstream robot planning. Code will be available at https://github.com/centiLinda/AToM-human-prediction.git.

Figures (6)

• Figure 1: After repeated interactions, the human develops confidence to move in a closer and more efficient manner. The robot, on the other hand, remains inefficient using a non-adaptive human model. Our proposed adaptive Theory-of-Mind (AToM) captures evolving human internal beliefs of others, allowing the robot to plan a more efficient path.
• Figure 2: Quantitative comparisons for Scenario 1 and Scenario 2. The simulation setup is illustrated on the left. We compare the prediction accuracy using ADE, and the efficiency and safety in resulting robot plans using Detour and Minimum Distance.
• Figure 3: Quantitative comparison between AToM and SF in Scenario 3. We compare the number of steps the robot takes to reach the goal which reflects the efficiency. SF leads to collisions in 7 rounds, which we highlighted using red crosses.
• Figure 4: Comparison between the predicted and ground truth trajectories in the first, the middle, and the last round of Scenario 2. Predictions are plotted at the second timestep.
• Figure 5: On the left, we compare the predicted trajectories in a sample round from Scenario 2. On the right, we plot three consecutive steps from Scenario 3, where SF prediction misleads the robot into a collision highlighted by the red circle.