Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Roles of LLMs in Planning: Embedding LLMs into Planning Graphs

Hankz Hankui Zhuo, Xin Chen, Rong Pan

TL;DR

The paper addresses the gap in planning with LLMs by embedding Large Language Models into a classical graph-based planner (Graphplan) through LLMs4Plan, which operates at two levels: proposing promising actions during forward graph expansion and selecting non-mutual action sets during backtracking. By guiding action generation and constraint-aware backtracking with targeted prompts and mutex reasoning, the approach enhances both solution quality and search efficiency, as shown across ten planning domains. Key contributions include a detailed two-phase integration, empirical evidence of improved success rates and reduced DFS backtracking, and an analysis of pruning-versus-sorting trade-offs. The work demonstrates that LLMs are most effective when used to augment, not replace, off-the-shelf planning techniques, offering a scalable path to integrating reasoning modules with planning frameworks in practice, while also outlining avenues for runtime improvements and broader framework adoption.

Abstract

Plan synthesis aims to generate a course of actions or policies to transit given initial states to goal states, provided domain models that could be designed by experts or learnt from training data or interactions with the world. Intrigued by the claims of emergent planning capabilities in large language models (LLMs), works have been proposed to investigate the planning effectiveness of LLMs, without considering any utilization of off-the-shelf planning techniques in LLMs. In this paper, we aim to further study the insight of the planning capability of LLMs by investigating the roles of LLMs in off-the-shelf planning frameworks. To do this, we investigate the effectiveness of embedding LLMs into one of the well-known planning frameworks, graph-based planning, proposing a novel LLMs-based planning framework with LLMs embedded in two levels of planning graphs, i.e., mutual constraints generation level and constraints solving level. We empirically exhibit the effectiveness of our proposed framework in various planning domains.

On the Roles of LLMs in Planning: Embedding LLMs into Planning Graphs

TL;DR

The paper addresses the gap in planning with LLMs by embedding Large Language Models into a classical graph-based planner (Graphplan) through LLMs4Plan, which operates at two levels: proposing promising actions during forward graph expansion and selecting non-mutual action sets during backtracking. By guiding action generation and constraint-aware backtracking with targeted prompts and mutex reasoning, the approach enhances both solution quality and search efficiency, as shown across ten planning domains. Key contributions include a detailed two-phase integration, empirical evidence of improved success rates and reduced DFS backtracking, and an analysis of pruning-versus-sorting trade-offs. The work demonstrates that LLMs are most effective when used to augment, not replace, off-the-shelf planning techniques, offering a scalable path to integrating reasoning modules with planning frameworks in practice, while also outlining avenues for runtime improvements and broader framework adoption.

Abstract

Plan synthesis aims to generate a course of actions or policies to transit given initial states to goal states, provided domain models that could be designed by experts or learnt from training data or interactions with the world. Intrigued by the claims of emergent planning capabilities in large language models (LLMs), works have been proposed to investigate the planning effectiveness of LLMs, without considering any utilization of off-the-shelf planning techniques in LLMs. In this paper, we aim to further study the insight of the planning capability of LLMs by investigating the roles of LLMs in off-the-shelf planning frameworks. To do this, we investigate the effectiveness of embedding LLMs into one of the well-known planning frameworks, graph-based planning, proposing a novel LLMs-based planning framework with LLMs embedded in two levels of planning graphs, i.e., mutual constraints generation level and constraints solving level. We empirically exhibit the effectiveness of our proposed framework in various planning domains.
Paper Structure (18 sections, 7 figures, 4 tables, 1 algorithm)

This paper contains 18 sections, 7 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Problem Formulation
  4. Our LLMs4Plan approach
  5. Building Planning Graphs with LLMs
  6. Building Mutual Constraints
  7. Sort Action Sets with LLMs and Search Solutions
  8. Experiment
  9. Experimental Setup
  10. Experimental Metrics
  11. Experimental Results
  12. Experimental Analysis
  13. Conclusion and Future Work
  14. Appendix / supplemental material
  15. Description of Testing Domains
  16. ...and 3 more sections

Figures (7)

  • Figure 1: Two critical steps in graph planning
  • Figure 2: The framework of a planning graph
  • Figure 3: The prompt for pruning actions
  • Figure 4: Mutual exclusion of actions
  • Figure 5: An example of planning graph and mutual constraints indicated in RED arcs
  • ...and 2 more figures