PIP-LLM: Integrating PDDL-Integer Programming with LLMs for Coordinating Multi-Robot Teams Using Natural Language

TL;DR

This work tackles translating natural-language instructions into executable, scalable multi-robot plans. It introduces PIP-LLM, a two-tier framework that uses LLMs to produce a team-level PDDL plan and an IP-based subtask allocation to robots, connected via a dependency graph. The approach integrates PDDL for team planning with an optimization-based MRTA that accounts for travel costs, workloads, and constraints, while a debugger/verifier improves plan correctness. Experiments in AI2-THOR and Gazebo demonstrate improved success rate, lower travel costs, and better load balancing compared with baselines, including ablations. The work offers a scalable path for language-driven coordination and is validated on both simulation and hardware, with code and dataset to be released.

Abstract

Enabling robot teams to execute natural language commands requires translating high-level instructions into feasible, efficient multi-robot plans. While Large Language Models (LLMs) combined with Planning Domain Description Language (PDDL) offer promise for single-robot scenarios, existing approaches struggle with multi-robot coordination due to brittle task decomposition, poor scalability, and low coordination efficiency. We introduce PIP-LLM, a language-based coordination framework that consists of PDDL-based team-level planning and Integer Programming (IP) based robot-level planning. PIP-LLMs first decomposes the command by translating the command into a team-level PDDL problem and solves it to obtain a team-level plan, abstracting away robot assignment. Each team-level action represents a subtask to be finished by the team. Next, this plan is translated into a dependency graph representing the subtasks' dependency structure. Such a dependency graph is then used to guide the robot-level planning, in which each subtask node will be formulated as an IP-based task allocation problem, explicitly optimizing travel costs and workload while respecting robot capabilities and user-defined constraints. This separation of planning from assignment allows PIP-LLM to avoid the pitfalls of syntax-based decomposition and scale to larger teams. Experiments across diverse tasks show that PIP-LLM improves plan success rate, reduces maximum and average travel costs, and achieves better load balancing compared to state-of-the-art baselines.

Figures (5)

• Figure 1: Overview of PIP-LLM. Users provide the task, context, and PDDL domain description in natural language (orange box). Then, LLMs will analyze the relevant information to construct a PDDL problem instance (PDDL Problem Generator). Next, the result is fed to a Debugger and a Verifier, which will check the syntax of PDDL problems and the correctness of the solution and feed the information back to the problem generation LLM. The generation process may be repeated several rounds. Each action in the output plan corresponds to a subtask that the robots as a team can do. Then, PIP-LLM uses this plan to construct a dependency graph to identify the dependencies between subtasks and facilitate parallel execution, which will then be used to allocate and schedule robots to finish all sub-tasks (green box, robot-level planning).
• Figure 2: An illustrative example to show the differences between our framework, PIP-LLM, and baselines (SMART-LLM and LaMMA-P). The task is put potato and knife in sinkbasin and trash the bottle and cup. (a)-(d) show the execution of results from PIP-LLM. (e)-(h) show the execution of baselines. The top left of (a) and (e) shows the allocated tasks of all robots. The top of (d) and (h) shows the cost of all robots. Ours achieves much smaller maximum/average cost. Red, blue, and green colors correspond to robots 1, 2, and 3, respectively.
• Figure 3: A team of 12 robots conducting tasks in the warehouse. Products 1, 2, 3, and 4 are in red, blue, green, and magenta, respectively. The task command is There should be full of product 1 in shelf 1; full of product 2 in shelf 2; full of product 3 in shelf 3; and shelf 4 should be empty. The team-level action plan is 0.0: (move-product shelf2 shelf3 product3 magnitude6) 1.0: (move-product shelf3 shelf2 product2 magnitude6), ..., 8.0: (move-product shelf4 shelf2 product2 magnitude4) (a)-(d) show the changes in product locations over time.
• Figure 4: PDDL domain used for the Gazebo warehouse environment.
• Figure 5: Snapshots of hardware experiments. The task is Push boxes to their corresponding goals. The planned paths for robots are marked as blue, green, and red dots, respectively, in (a). Three robots are available for the task. The transparent robots and boxes along the paths denote their future locations.