Using Large Language Models for Abstraction of Planning Domains - Extended Version

TL;DR

The paper tackles the challenge of generating useful abstractions of planning domains from concrete PDDL models using in-context learning with large language models. It formalizes Planning Domain Abstraction Generation (PDAG) by leveraging refinement mappings and m-bisimulation to relate high-level and low-level action theories, with soundness and completeness criteria. It then presents four abstraction categories (alternative actions, action sequences, and parameter abstractions, plus combinations) and demonstrates through multiple examples (e.g., travel arrangements, cloud apps, hotel bookings, and event planning) how GPT-4o can synthesize abstract domains and problems that are subsequently validated by symbolic tools and experts. The evaluation shows promising results in simple settings, highlights common pitfalls (e.g., over-generalization of predicates and syntax errors in more complex abstractions), and outlines future work on formal validation, ADL extensions, and automated soundness checks to improve reliability and applicability in planning systems.

Abstract

Generating an abstraction of a dynamic domain that aligns with a given purpose remains a significant challenge given that the choice of such an abstraction can impact an agent's ability to plan, reason, and provide explanations effectively. We model the agent's concrete behaviors in PDDL and investigate the use of in-context learning with large language models (LLMs) for the generation of abstract PDDL domains and problem instances, given an abstraction objective specified in natural language. The benchmark examples we use are new and have not been part of the data any LLMs have been trained on. We consider three categories of abstractions: abstraction of choice of alternative concrete actions, abstraction of sequences of concrete actions, and abstraction of action/predicate parameters, as well as combinations of these. The generated abstract PDDL domains and problem instances are then checked by symbolic validation tools as well as human experts. Our experiments show that GPT-4o can generally synthesize useful planning domain abstractions in simple settings, although it is better at abstracting over actions than over the associated fluents.

Figures (15)

• Figure 1: Generating Abstract PDDL Domain and Problem Instance
• Figure 2: Abstraction of Alternative Concrete Actions: Zero-Shot Learning, System Prompt Part 1
• Figure 3: Abstraction of Alternative Concrete Actions: Zero-Shot Learning, System Prompt Part 2
• Figure 4: Abstraction of Alternative Concrete Actions: Zero-Shot Learning, User Prompt (User Query)
• Figure 5: Abstraction of Alternative Concrete Actions: One-Shot Learning, System Prompt