From Real World to Logic and Back: Learning Generalizable Relational Concepts For Long Horizon Robot Planning

TL;DR

This work addresses the challenge of enabling robots to generalize from limited demonstrations to long-horizon planning in unseen environments by autonomously inventing symbolic relational concepts and world models. The authors introduce LAMP, which learns relational critical regions from raw trajectories, constructs a symbolic vocabulary, and invents high-level actions and interpreters to form a transferable PDDL-like world model, enabling zero-shot solving of tasks far beyond training. Key contributions include the Relational Critical Regions framework, the Relation Inventor for semantic predicates, and the Action Inventor for automatic action models, which together yield interpretable, scalable planning that transfers across domains and object counts, achieving up to 18× generalization and competitive performance with hand-crafted baselines. The results demonstrate substantial gains in sample efficiency and generalization, with code and data released to support reuse; future work targets stochastic environments, broader object categories, and robust perception integration.

Abstract

Robots still lag behind humans in their ability to generalize from limited experience, particularly when transferring learned behaviors to long-horizon tasks in unseen environments. We present the first method that enables robots to autonomously invent symbolic, relational concepts directly from a small number of raw, unsegmented, and unannotated demonstrations. From these, the robot learns logic-based world models that support zero-shot generalization to tasks of far greater complexity than those in training. Our framework achieves performance on par with hand-engineered symbolic models, while scaling to execution horizons far beyond training and handling up to 18$\times$ more objects than seen during learning. The results demonstrate a framework for autonomously acquiring transferable symbolic abstractions from raw robot experience, contributing toward the development of interpretable, scalable, and generalizable robot planning systems. Project website and code: https://aair-lab.github.io/r2l-lamp.

Figures (10)

• Figure 1: Example of transfer achieved in this paper. (a) shows the complete set of training tasks for one of our experiment domains; (b) shows the automatically invented symbolic concepts and world models by our approach; (c) shows some of the test tasks zero-shot solved using the learned model despite significantly greater numbers and complexity. Interpretable relation names are provided for clarity, not learned by the method.
• Figure 2: Overview of LAMP. From unlabeled, unsegmented demonstrations (a), LAMP learns relational critical region predictors (b), each defining relations between object pairs (c). These relations form abstract states, with learned actions as transitions (d), together constituting a symbolic world model learned from scratch. For new tasks with more objects or obstructions, LAMP predicts new relations to build the abstract state space and goal, then uses the model to synthesize behaviors that achieve it (f).
• Figure 3: Examples of relations and an invented action with their critical regions. (a) Each image shows a binary predicate with its semantic interpretation; red dots indicate sampled poses within the relational critical region. (b) An auto-invented action is shown with parameters, preconditions, and effects. Relation names in blue reflect author-provided interpretations.
• Figure 4: Empirical evaluation of LAMP. (a) Generalization achieved by our approach, with the x-axis denoting domain and the y-axis the generalization factor. The red dotted line marks the 1× generalization zone typical of imitation learning or behavior cloning. (b) Robustness comparison with Code-as-Policies and STAMP, where the x-axis shows the number of training demonstrations and the y-axis the proportion of solved test tasks; shaded regions indicate standard deviation over 10 runs. (c) Training tasks used to learn symbolic world models. (d) Test tasks used to evaluate generalization.
• Figure 5: Overview of the proposed algorithm. The approach comprises of two core components: $(i)$ "Relation Inventor" that uses training demonstrations and generates novel relational vocabularies for robots, and $(ii)$ "Action Inventor" that invents high-level symbolic actions. Together, they enable zero-shot goal-achieving behavior by integrating relational semantics, predictive action models, and hybrid planning.

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3