General Dynamic Goal Recognition using Goal-Conditioned and Meta Reinforcement Learning

TL;DR

This work addresses Goal Recognition under dynamic, cross-domain conditions by formalizing General Dynamic Goal Recognition (GDGR) and proposing Adaptive Universal Recognition Algorithm (AURA). It presents two RL-based instantiations, GC-AURA (goal-conditioned RL within a fixed domain) and Meta-AURA (meta-RL across multiple domains), to achieve rapid adaptation to new goals and environments in real time. Extensive experiments across MiniGrid, Point-Maze, and Panda-Gym demonstrate that AURA reduces adaptation times and maintains high recognition accuracy under noise and partial observability, with GC-AURA excelling at continuous-goal generalization and Meta-AURA enabling swift cross-domain transfer. Collectively, the work advances GR from static, single-domain settings to dynamic, real-world-like scenarios, offering a scalable framework for real-time goal inference in autonomous and assistive systems.

Abstract

Understanding an agent's goal through its behavior is a common AI problem called Goal Recognition (GR). This task becomes particularly challenging in dynamic environments where goals are numerous and ever-changing. We introduce the General Dynamic Goal Recognition (GDGR) problem, a broader definition of GR aimed at real-time adaptation of GR systems. This paper presents two novel approaches to tackle GDGR: (1) GC-AURA, generalizing to new goals using Model-Free Goal-Conditioned Reinforcement Learning, and (2) Meta-AURA, adapting to novel environments with Meta-Reinforcement Learning. We evaluate these methods across diverse environments, demonstrating their ability to achieve rapid adaptation and high GR accuracy under dynamic and noisy conditions. This work is a significant step forward in enabling GR in dynamic and unpredictable real-world environments.

Figures (12)

• Figure 1: The General Dynamic Goal Recognition Problem
• Figure 2: Adaptive Universal Recognition Algorithm (AURA).
• Figure 3: Training curves for the PandaReach-v3 domain. The orange curve represents the Goal-Conditioned TRPO policy (for GC-AURA) trained across all goals in the continuous goal space. The blue curve shows the goal-directed TRPO policy (for DRACO) trained for a specific goal. The shaded regions indicate variations across changing seeds and goals.
• Figure 4: F-Score for PandaReach environment with 10% observability rate, evaluated across 20 different GR problems. The orange curve represents the F-Score of the GC-TRPO policy (for GC-AURA) across different iterations. The blue curve shows the F-Score of the goal-directed TRPO policy (for DRACO) across different iterations. The shaded regions indicate the standard deviations across the 20 GR problems.
• Figure 5: Examples from the train set of Point-Maze environments for MAML-RL training

Theorems & Definitions (2)

• Definition 1
• Definition 2