Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Vote-Tree-Planner: Optimizing Execution Order in LLM-based Task Planning Pipeline via Voting

Chaoyuan Zhang, Zhaowei Li, Wentao Yuan

TL;DR

The paper tackles inefficiencies in LLM-based robotic task planning by reducing redundant queries and increasing plan executability. It presents Vote-Tree-Planner, a multi-stage pipeline that fuses Prog-Prompt formatting with a voting-based execution tree to guide plan traversal. Key contributions include a unique command extractor, a reordering-based plan generation step, and a Vote-Tree that aggregates plans by shared prefixes to improve reliability and reduce LLM queries. Experiments in the Virtual Home domain show gains in success rate and goal recall with comparable executability, while qualitative analysis highlights more concise, non-redundant plans. The work advances practical, efficient, and robust LLM-driven planning with potential for broader adoption in embodied AI tasks.

Abstract

Integrating large language models (LLMs) into closed-loop robotic task planning has become increasingly popular within embodied artificial intelligence. Previous efforts mainly focused on leveraging the strong reasoning abilities of LLMs to enhance task planning performance while often overlooking task planning efficiency and executability due to repetitive queries to LLMs. This paper addresses the synergy between LLMs and task planning systems, aiming to minimize redundancy while enhancing planning effectiveness. Specifically, building upon Prog-Prompt and the high-level concept of Tree-Planner, we propose Vote-Tree-Planner. This sampling strategy utilizes votes to guide plan traversal during the decision-making process. Our approach is motivated by a straightforward observation: assigning weights to agents during decision-making enables the evaluation of critical paths before execution. With this simple vote-tree construction, our method further improves the success rate and reduces the number of queries to LLMs. The experimental results highlight that our Vote-Tree-Planner demonstrates greater stability and shows a higher average success rate and goal condition recall on the unseen dataset compared with previous baseline methods. These findings underscore the potential of the Vote-Tree-Planner to enhance planning accuracy, reliability, and efficiency in LLM-based planning systems.

Vote-Tree-Planner: Optimizing Execution Order in LLM-based Task Planning Pipeline via Voting

TL;DR

The paper tackles inefficiencies in LLM-based robotic task planning by reducing redundant queries and increasing plan executability. It presents Vote-Tree-Planner, a multi-stage pipeline that fuses Prog-Prompt formatting with a voting-based execution tree to guide plan traversal. Key contributions include a unique command extractor, a reordering-based plan generation step, and a Vote-Tree that aggregates plans by shared prefixes to improve reliability and reduce LLM queries. Experiments in the Virtual Home domain show gains in success rate and goal recall with comparable executability, while qualitative analysis highlights more concise, non-redundant plans. The work advances practical, efficient, and robust LLM-driven planning with potential for broader adoption in embodied AI tasks.

Abstract

Integrating large language models (LLMs) into closed-loop robotic task planning has become increasingly popular within embodied artificial intelligence. Previous efforts mainly focused on leveraging the strong reasoning abilities of LLMs to enhance task planning performance while often overlooking task planning efficiency and executability due to repetitive queries to LLMs. This paper addresses the synergy between LLMs and task planning systems, aiming to minimize redundancy while enhancing planning effectiveness. Specifically, building upon Prog-Prompt and the high-level concept of Tree-Planner, we propose Vote-Tree-Planner. This sampling strategy utilizes votes to guide plan traversal during the decision-making process. Our approach is motivated by a straightforward observation: assigning weights to agents during decision-making enables the evaluation of critical paths before execution. With this simple vote-tree construction, our method further improves the success rate and reduces the number of queries to LLMs. The experimental results highlight that our Vote-Tree-Planner demonstrates greater stability and shows a higher average success rate and goal condition recall on the unseen dataset compared with previous baseline methods. These findings underscore the potential of the Vote-Tree-Planner to enhance planning accuracy, reliability, and efficiency in LLM-based planning systems.

Paper Structure

This paper contains 21 sections, 6 equations, 3 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Definition
  4. Proposed Method: Vote-Tree-Planner
  5. Prog-Prompt Formatting
  6. Plan Generation and Unique Command Extraction
  7. Reordering Prompt Formatting and Sequence Generation
  8. Vote-Tree Construction
  9. Execution
  10. Experiment
  11. Experimental Setup
  12. Environment
  13. Dataset
  14. Evaluation Metrics
  15. Baselines
  16. ...and 6 more sections

Figures (3)

  • Figure 1: Illustration of our proposed Vote-Tree-Planner pipeline. $N$ plans are generated by the framework of Prog-Prompt using the LLM, then the unique commands are extracted and reordered into $M$ plans by the LLM again. These plans are then merged into an execution tree as inspired by Tree-Planner hu2023treeplanner. The execution tree is then tested in the virtual home simulator puig2018virtualhomepuig2020watchandhelp. We integrate the main characteristics from both pipelines to form our proposed method. Left (the Prog-Prompt Block): prompts the LLM and generates $N$ plans, then unique commands are extracted to a set. Middle (left column of the Command Reordering and Tree Construction Block): prompts the LLM again using the unique command set and generates $M$ plans. Right (right column of the Command Reordering and Tree Construction Block): constructs the generated plans into a tree with votes, and then executes the plan according to the votes.
  • Figure 2: Illustration of voting mechanism. Error Correction (left): When a node is successfully executed. The next step is finding the child of that node with the highest vote and trying to execute the command of that node. If the execution fails, we execute the second-highest-voted child. If all the children fail, the mechanism should traverse another branch according to the number of votes and continue. Successful Execution (right): The execution process terminates when a node without any child nodes is executed successfully.
  • Figure 3: Qualitative analysis. The two planning examples, "Microwave Salmon" (left) and "Put Salmon In The Fridge" (right), show the comparison between two plans generated by Prog-Prompt singh2022progprompt (on the left of each side) and our method (on the right of each side). Commands in red represent what Prog-Prompt did wrong and redundantly, while our method did correctly and concisely. Commands in orange represent what Prog-Prompt did redundantly while our method did concisely. Commands in green represent necessary and correct commands for both Prog-Prompt and our method.