Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Planning with Multi-Constraints via Collaborative Language Agents

Cong Zhang, Derrick Goh Xin Deik, Dexun Li, Hao Zhang, Yong Liu

TL;DR

PMC tackles planning under multiple constraints for LLM-based agents by introducing a hierarchical, zero-shot collaboration among a manager, executors, a supervisor, and a deliverer. It decomposes tasks into subtasks arranged as a dependency graph and maps each subtask to executable tool calls, enabling effective management of local and global constraints. Empirical results on TravelPlanner and API-Bank show substantial gains over GPT-4-based baselines and demonstrate viability with smaller open-source LLMs like LLaMA-3.1-8B, highlighting PMC's potential to scale real-world constrained planning with collaborative agents. The framework’s divide-and-conquer strategy improves planning stability and interpretability, making it a promising direction for robust, constraint-aware LLM-based automation.

Abstract

The rapid advancement of neural language models has sparked a new surge of intelligent agent research. Unlike traditional agents, large language model-based agents (LLM agents) have emerged as a promising paradigm for achieving artificial general intelligence (AGI) due to their superior reasoning and generalization capabilities. Effective planning is crucial for the success of LLM agents in real-world tasks, making it a highly pursued topic in the community. Current planning methods typically translate tasks into executable action sequences. However, determining a feasible or optimal sequence for complex tasks with multiple constraints at fine granularity, which often requires compositing long chains of heterogeneous actions, remains challenging. This paper introduces Planning with Multi-Constraints (PMC), a zero-shot methodology for collaborative LLM-based multi-agent systems that simplifies complex task planning with constraints by decomposing it into a hierarchy of subordinate tasks. Each subtask is then mapped into executable actions. PMC was assessed on two constraint-intensive benchmarks, TravelPlanner and API-Bank. Notably, PMC achieved an average 42.68% success rate on TravelPlanner, significantly higher than GPT-4 (2.92%), and outperforming GPT-4 with ReAct on API-Bank by 13.64%, showing the immense potential of integrating LLM with multi-agent systems. We also show that PMC works with small LLM as the planning core, e.g., LLaMA-3.1-8B.

Planning with Multi-Constraints via Collaborative Language Agents

TL;DR

PMC tackles planning under multiple constraints for LLM-based agents by introducing a hierarchical, zero-shot collaboration among a manager, executors, a supervisor, and a deliverer. It decomposes tasks into subtasks arranged as a dependency graph and maps each subtask to executable tool calls, enabling effective management of local and global constraints. Empirical results on TravelPlanner and API-Bank show substantial gains over GPT-4-based baselines and demonstrate viability with smaller open-source LLMs like LLaMA-3.1-8B, highlighting PMC's potential to scale real-world constrained planning with collaborative agents. The framework’s divide-and-conquer strategy improves planning stability and interpretability, making it a promising direction for robust, constraint-aware LLM-based automation.

Abstract

The rapid advancement of neural language models has sparked a new surge of intelligent agent research. Unlike traditional agents, large language model-based agents (LLM agents) have emerged as a promising paradigm for achieving artificial general intelligence (AGI) due to their superior reasoning and generalization capabilities. Effective planning is crucial for the success of LLM agents in real-world tasks, making it a highly pursued topic in the community. Current planning methods typically translate tasks into executable action sequences. However, determining a feasible or optimal sequence for complex tasks with multiple constraints at fine granularity, which often requires compositing long chains of heterogeneous actions, remains challenging. This paper introduces Planning with Multi-Constraints (PMC), a zero-shot methodology for collaborative LLM-based multi-agent systems that simplifies complex task planning with constraints by decomposing it into a hierarchy of subordinate tasks. Each subtask is then mapped into executable actions. PMC was assessed on two constraint-intensive benchmarks, TravelPlanner and API-Bank. Notably, PMC achieved an average 42.68% success rate on TravelPlanner, significantly higher than GPT-4 (2.92%), and outperforming GPT-4 with ReAct on API-Bank by 13.64%, showing the immense potential of integrating LLM with multi-agent systems. We also show that PMC works with small LLM as the planning core, e.g., LLaMA-3.1-8B.
Paper Structure (38 sections, 2 theorems, 8 figures, 7 tables)

This paper contains 38 sections, 2 theorems, 8 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Literature Review for Language Model Agent Planning
  3. Preliminaries
  4. Planning with Multi-Constraints
  5. The Collaborative Multi-Agent System Design
  6. Manager Agent
  7. Executor Agent
  8. Supervisor Agent
  9. Deliverer Agent
  10. Hierarchical Task Planning and Execution
  11. Task-Level Planning.
  12. Step-Level Planning and Execution.
  13. Experiments
  14. Experiment Setup
  15. Benchmarks
  16. ...and 23 more sections

Key Result

Proposition 4.4

A sub-task$T_i$ is accomplishable while adhering to local constraints if and only if all the sub-tasks within its direct neighborhood $\mathcal{N}(T_i)$ are accomplished with their respective local constraints maintained.

Figures (8)

  • Figure 1: An overview of PMC Framework. The PMC Framework provides a structured methodology for managing and executing sub-tasks within a directed sub-task graph topology, as the manager coordinates. For instance, the completion of $\text{Task}_2$ depends on the outputs derived from $\text{Task}_1$, which a supervisor agent subsequently consolidates. The executor agent is tasked with implementing the sub-task, considering any local constraints present. Upon completion of the sub-tasks, the deliverer agent is responsible for aggregating all sub-task outcomes to satisfy global constraints and subsequently achieve the overarching task objectives.
  • Figure 2: The zero-shot prompt structure for each agent. Note both supervisor and deliverer agents do not require function calls, while different executors will have different tool lists.
  • Figure 3: An overview of sub-task graph, which reveals the task-level decomposition. The manager agent decomposes the main task into several sub-tasks with inter-dependencies (dashed arrows).
  • Figure 4: step-level Planning and Execution. The executor is furnished with a planning core and a toolbox comprising diverse functions. This includes an off-the-shelf planning algorithm such as ReAct yao2023react, which is used to translate the sub-task into a series of executable function calls required to accomplish the assigned sub-task.
  • Figure 5: Step 1 of prompt optimization for Manager Agent. Full demonstration provided. This demonstration above consists of two sub-tasks, the number of sub-tasks will vary based on the main task. The conversion of prompt into zero-shot is shown in Figure \ref{['fig:demo-2']}, \ref{['fig:demo-3']} and \ref{['fig:demo-4']}.
  • ...and 3 more figures

Theorems & Definitions (5)

  • Definition 4.1
  • Definition 4.2
  • Definition 4.3
  • Proposition 4.4
  • Proposition D.1