Improving Large Language Model Planning with Action Sequence Similarity

TL;DR

This work tackles the problem of enhancing large language model planning by improving exemplar selection in in-context learning. It introduces Action Sequence Similarity (AS), based on the longest common ordered action sequence, as a robust signal to select exemplars, and presents GRASE-DC, a two-stage pipeline that generatively re-samples high-AS exemplars (GRASE) and dynamically clusters them (DC) to balance relevance and diversity. Empirical results across classical PDDL planning tasks and natural-language trip planning show substantial planning accuracy gains (up to 11–40 points) with fewer exemplars, and further improvements when combined with a validator (GRASE-DC* + VAL). The approach generalizes across backbone LLMs and supports out-of-distribution problems, with efficiency analyses and approximations (MLP, BPE-Proxy) offering practical trade-offs. Overall, AS-based exemplar selection provides a principled and scalable way to boost LLM planning in diverse settings.

Abstract

Planning is essential for artificial intelligence systems to look ahead and proactively determine a course of actions to reach objectives in the virtual and real world. Recent work on large language models (LLMs) sheds light on their planning capability in various tasks. However, it remains unclear what signals in the context influence the model performance. In this work, we explore how to improve the model planning capability through in-context learning (ICL), specifically, what signals can help select the exemplars. Through extensive experiments, we observe that commonly used problem similarity may result in false positives with drastically different plans, which can mislead the model. In response, we propose to sample and filter exemplars leveraging plan side action sequence similarity (AS). We propose GRASE-DC: a two-stage pipeline that first re-samples high AS exemplars and then curates the selected exemplars with dynamic clustering on AS to achieve a balance of relevance and diversity. Our experimental result confirms that GRASE-DC achieves significant performance improvement on various planning tasks (up to ~11-40 point absolute accuracy improvement with 27.3% fewer exemplars needed on average). With GRASE-DC* + VAL, where we iteratively apply GRASE-DC with a validator, we are able to even boost the performance by 18.9% more. Extensive analysis validates the consistent performance improvement of GRASE-DC with various backbone LLMs and on both classical planning and natural language planning benchmarks. GRASE-DC can further boost the planning accuracy by ~24 absolute points on harder problems using simpler problems as exemplars over a random baseline. This demonstrates its ability to generalize to out-of-distribution problems.

Figures (11)

• Figure 1: An illustration of our two-stage GRASE-DC pipeline. Given a test example, [GRASE stage:] we first use the random exemplars from the candidate pool to acquire the initial model plan. We then utilize actions in this plan to rank the pool with the action sequence similarity. [DC stage:] we further sample the specific exemplar pool for the test example with the relevance and diversity in the lens of clusters based on action sequence similarity. Finally, we conduct ICL and prompt LLMs with the sampled pool and original test example. We can iteratively apply GRASE-DC, i.e., GRASE-DC$^*$, by re-sampling exemplars with the action sequences of the generated plans.
• Figure 2: PDDL Planning on various tasks with Gemini 1.5 Pro. $\text{Baseline}_{AS}$ denotes ranking exemplars with the plan similarity given Oracle test plans. Task denotes the baseline that calculates the similarity between each test example and exemplar candidate with a token overlap in descriptions.
• Figure 3: PDDL planning accuracy on various tasks with Gemini 1.5 Pro. $\text{Baseline}_{AS}$ (AS lines) denotes the use of Oracle test plans. $\text{Baseline}_{AS}$ and Random lines are the anchors across rows (the second row focuses on exemplars from 4 to 40). GRASE denotes the use of model output plans from random exemplars. VAL denotes the use of the plan validator. GRASE-DC$^*$ denotes another iteration with the model outputs from GRASE-DC. Numbers of exemplars for GRASE-DC and its iteration denote the average number of exemplars used over the whole test set with $N_c=1,2,3$.
• Figure 4: PDDL Planning on Blocksworld with various LLMs with different numbers of exemplars. All models are using the same set of exemplars for Random. Opus denotes Claude-3.0-Opus.
• Figure 5: Natural language planning performance on Trip Planning with Gemini 1.5 Pro. (left) ICL performance with different numbers of exemplars and signals for exemplars sampling. $\text{Baseline}_{AS}$ (denoted as AS in the figure) denotes the use of Oracle test plans. (right) ICL performance with different problem complexity (denoted by number of cities). Best Performing denotes the use of 40 and 100 exemplars for Random and GRASE, respectively.