Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

IMR-LLM: Industrial Multi-Robot Task Planning and Program Generation using Large Language Models

Xiangyu Su, Juzhan Xu, Oliver van Kaick, Kai Xu, Ruizhen Hu

TL;DR

The proposed IMR-LLM is a novel LLM-driven Industrial Multi-Robot task planning and program generation framework that utilizes LLMs to assist in constructing disjunctive graphs and employ deterministic solving methods to obtain a feasible and efficient high-level task plan.

Abstract

In modern industrial production, multiple robots often collaborate to complete complex manufacturing tasks. Large language models (LLMs), with their strong reasoning capabilities, have shown potential in coordinating robots for simple household and manipulation tasks. However, in industrial scenarios, stricter sequential constraints and more complex dependencies within tasks present new challenges for LLMs. To address this, we propose IMR-LLM, a novel LLM-driven Industrial Multi-Robot task planning and program generation framework. Specifically, we utilize LLMs to assist in constructing disjunctive graphs and employ deterministic solving methods to obtain a feasible and efficient high-level task plan. Based on this, we use a process tree to guide LLMs to generate executable low-level programs. Additionally, we create IMR-Bench, a challenging benchmark that encompasses multi-robot industrial tasks across three levels of complexity. Experimental results indicate that our method significantly surpasses existing methods across all evaluation metrics.

IMR-LLM: Industrial Multi-Robot Task Planning and Program Generation using Large Language Models

TL;DR

The proposed IMR-LLM is a novel LLM-driven Industrial Multi-Robot task planning and program generation framework that utilizes LLMs to assist in constructing disjunctive graphs and employ deterministic solving methods to obtain a feasible and efficient high-level task plan.

Abstract

In modern industrial production, multiple robots often collaborate to complete complex manufacturing tasks. Large language models (LLMs), with their strong reasoning capabilities, have shown potential in coordinating robots for simple household and manipulation tasks. However, in industrial scenarios, stricter sequential constraints and more complex dependencies within tasks present new challenges for LLMs. To address this, we propose IMR-LLM, a novel LLM-driven Industrial Multi-Robot task planning and program generation framework. Specifically, we utilize LLMs to assist in constructing disjunctive graphs and employ deterministic solving methods to obtain a feasible and efficient high-level task plan. Based on this, we use a process tree to guide LLMs to generate executable low-level programs. Additionally, we create IMR-Bench, a challenging benchmark that encompasses multi-robot industrial tasks across three levels of complexity. Experimental results indicate that our method significantly surpasses existing methods across all evaluation metrics.
Paper Structure (16 sections, 2 equations, 5 figures, 2 tables)

This paper contains 16 sections, 2 equations, 5 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Multi-Robot Task Planning
  4. Robot Program Generation
  5. DEFINITIONS
  6. Disjunctive Graph in Robotic Job-Shop Scheduling
  7. Problem Formulation
  8. METHOD
  9. Task Planning
  10. Program Generation
  11. Execution
  12. EXPERIMENTS
  13. Benchmark
  14. Simulation Experiments
  15. Real-world Experiments
  16. ...and 1 more sections

Figures (5)

  • Figure 1: A multi-robot industrial production line. Our method transforms manufacturing tasks described in natural language into high-level task plans and low-level execution programs, allowing multiple robots to collaborate efficiently in completing the tasks.
  • Figure 2: An overview of our method. Given an instruction $I$, an industrial scene $S$, and program examples $\mathbb{E}$, our method performs task planning (highlighted in green) to decompose operations, assign robots, and schedule operations using a disjunctive graph and a heuristic solver. This is followed by program generation (highlighted in blue) that translates the plan into executable Python code under the guidance of an operation process tree. The resulting high-level plan and low-level program enable collaborative execution by multiple robots.
  • Figure 3: An overview of our dataset. Our tasks consist of (a) various scenes and (b) various machines and robots equipped with different end-effectors. (c) A pie chart showing the distribution of task types on the left and a bar chart showing the average number of operations per task type on the right.
  • Figure 4: Keyframes in the execution process. Workpiece 1 and 2 (highlighted in pink and green) are initially placed on the conveyor belt. The robots (highlighted in the yellow boxes) collaborate to complete the operations and finally place workpiece 2 on pallet 1 (keyframe 5) and workpiece 1 on pallet 2 (keyframe 6). Each keyframe contains textual annotations describing the operations performed by the robots.
  • Figure 5: A real-world experiment. Three robots collaborate to complete a transportation task.