GRAIL: Goal Recognition Alignment through Imitation Learning

TL;DR

GRAIL reframes goal recognition as a set of per-goal imitation-learning problems, training one policy per candidate goal offline and performing one-shot inference online by scoring the observed trajectory against each policy. By leveraging BC, GAIL, or AIRL, GRAIL captures suboptimal and systematically biased behavior, enabling fast, planner-free GR with robust performance across MiniGrid and PandaReach. Across biased-optimal, suboptimal, and optimal regimes, GRAIL demonstrates substantial F1-score gains and efficiency improvements over RL-based baselines, particularly under limited observability and larger goal sets. This approach offers a scalable, modular path toward robust AI alignment by interpreting agent goals in uncertain environments with real-time inference requirements.

Abstract

Understanding an agent's goals from its behavior is fundamental to aligning AI systems with human intentions. Existing goal recognition methods typically rely on an optimal goal-oriented policy representation, which may differ from the actor's true behavior and hinder the accurate recognition of their goal. To address this gap, this paper introduces Goal Recognition Alignment through Imitation Learning (GRAIL), which leverages imitation learning and inverse reinforcement learning to learn one goal-directed policy for each candidate goal directly from (potentially suboptimal) demonstration trajectories. By scoring an observed partial trajectory with each learned goal-directed policy in a single forward pass, GRAIL retains the one-shot inference capability of classical goal recognition while leveraging learned policies that can capture suboptimal and systematically biased behavior. Across the evaluated domains, GRAIL increases the F1-score by more than 0.5 under systematically biased optimal behavior, achieves gains of approximately 0.1-0.3 under suboptimal behavior, and yields improvements of up to 0.4 under noisy optimal trajectories, while remaining competitive in fully optimal settings. This work contributes toward scalable and robust models for interpreting agent goals in uncertain environments.

Figures (6)

• Figure 1: A navigation setup illustrating how ignoring systematic human biases like preference for park routes can lead to reduced inference accuracy in standard GR systems.
• Figure 2: A comparison between model-based GR approaches (top), the RL-based GR approaches (middle), and our proposed framework (bottom).
• Figure 3: Minigrid (9×9 grid, 7×7 free states) Q-learning visit counts per state (position + orientation) during training, shown as heatmaps with overlaid counts. Yellow indicates high visit frequency. (left) Training visits for goal at $(7,1)$. (right) Training visits for goal at $(7,7)$.
• Figure 4: F$_1$-score comparison on Minigrid with systematically biased optimal demonstrations and 2 goals under 20–40% observability across 10 runs. GRAIL variants (green/red/yellow) achieve perfect inference, compared to GRAQL (blue).
• Figure 5: $F_1$-score comparison on PandaReach with optimal and noisy demonstrations (uniform and gaussian) and 4 goals under 2% observability across 10 runs.

Theorems & Definitions (2)

• Definition 1
• Definition 2: Goal Recognition from Demonstrations (GRD)