Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models

Shyam Sundar Kannan, Vishnunandan L. N. Venkatesh, Byung-Cheol Min

TL;DR

SMART-LLM addresses the challenge of planning for heterogeneous robot teams under natural-language instructions. It uses a four-stage LLM-driven pipeline—task decomposition, coalition formation, task allocation, and execution—with Pythonic prompts to generate executable plans. A dedicated benchmark on AI2-THOR and evaluations in simulation and real robots demonstrate the method's scalability, generalization, and practical viability. The work shows that LLM-driven task planning can unify decomposition, coordination, and execution without task-specific coding.

Abstract

In this work, we introduce SMART-LLM, an innovative framework designed for embodied multi-robot task planning. SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models (LLMs), harnesses the power of LLMs to convert high-level task instructions provided as input into a multi-robot task plan. It accomplishes this by executing a series of stages, including task decomposition, coalition formation, and task allocation, all guided by programmatic LLM prompts within the few-shot prompting paradigm. We create a benchmark dataset designed for validating the multi-robot task planning problem, encompassing four distinct categories of high-level instructions that vary in task complexity. Our evaluation experiments span both simulation and real-world scenarios, demonstrating that the proposed model can achieve promising results for generating multi-robot task plans. The experimental videos, code, and datasets from the work can be found at https://sites.google.com/view/smart-llm/.

SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models

TL;DR

SMART-LLM addresses the challenge of planning for heterogeneous robot teams under natural-language instructions. It uses a four-stage LLM-driven pipeline—task decomposition, coalition formation, task allocation, and execution—with Pythonic prompts to generate executable plans. A dedicated benchmark on AI2-THOR and evaluations in simulation and real robots demonstrate the method's scalability, generalization, and practical viability. The work shows that LLM-driven task planning can unify decomposition, coordination, and execution without task-specific coding.

Abstract

In this work, we introduce SMART-LLM, an innovative framework designed for embodied multi-robot task planning. SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models (LLMs), harnesses the power of LLMs to convert high-level task instructions provided as input into a multi-robot task plan. It accomplishes this by executing a series of stages, including task decomposition, coalition formation, and task allocation, all guided by programmatic LLM prompts within the few-shot prompting paradigm. We create a benchmark dataset designed for validating the multi-robot task planning problem, encompassing four distinct categories of high-level instructions that vary in task complexity. Our evaluation experiments span both simulation and real-world scenarios, demonstrating that the proposed model can achieve promising results for generating multi-robot task plans. The experimental videos, code, and datasets from the work can be found at https://sites.google.com/view/smart-llm/.
Paper Structure (17 sections, 3 figures, 3 tables)

This paper contains 17 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Problem Formulation
  4. Methodology
  5. Stage 1: Task Decomposition
  6. Stage 2: Coalition Formation
  7. Stage 3: Task Allocation
  8. Stage 4: Task Execution
  9. Experiments
  10. Benchmark Dataset
  11. Simulation Experiments
  12. Real-Robot Experiments
  13. Evaluation Metrics
  14. Results and Discussion
  15. Simulation Experiments
  16. ...and 2 more sections

Figures (3)

  • Figure 1: An overview of SMART-LLM: Smart Multi-Agent Robot Task planning using Large Language Models (LLM). Given a high-level instruction, SMART-LLM decomposes the instruction into sub-tasks assigning them to individual robots based on their specific skills and capabilities, and orchestrating their execution in a coherent and logical sequence.
  • Figure 2: System overview: SMART-LLM consists of four key stages: i) Task Decomposition: a prompt consisting of robot skills, objects, and task decomposition samples is combined with the input instruction. This is then fed to the LLM model to decompose the input task; ii) Coalition Formation: a prompt consisting of a list of robots, objects available in the environment, sample decomposed task examples along with corresponding coalition policy describing the formation of robot teams for those tasks, and decomposed task plan for the input task from the previous stage, is given to the LLM, to generate a coalition policy for the input task; iii) Task Allocation: a prompt consisting of sample decomposed tasks, their coalition policy and allocated task plans based on the coalition policy is given to the LLM, along with coalition policy generated for the input task. The LLM then outputs an allocated task plan based on this information; and iv) Task Execution: based on the allocated code generated, the robot executes the tasks. “...” is used for brevity.
  • Figure 3: Real-robot experiment: a) team of robots and the regions to be patrolled; b) robots after task planning and patrolling their respective regions allocated based on visibility area.