LLM-HBT: Dynamic Behavior Tree Construction for Adaptive Coordination in Heterogeneous Robots

TL;DR

The paper tackles the challenge of robust, long-horizon coordination in heterogeneous multi-robot systems operating in dynamic environments. It introduces LLM-HBT, a framework that fuses large language model reasoning with behavior trees, guided by a centralized virtual allocator named Alex, to dynamically construct, extend, and synchronize task plans online. The approach combines local BT extensions with cross-robot task reallocation to ensure resilience and scalability, validated through 60 tasks in simulation across three heterogeneous scenarios and a real-world cafe deployment. Results show superior task success rates and reduced execution steps compared with baselines, illustrating the practical impact of integrating language-driven reasoning with modular BT structures for adaptable multi-robot coordination.

Abstract

We introduce a novel framework for automatic behavior tree (BT) construction in heterogeneous multi-robot systems, designed to address the challenges of adaptability and robustness in dynamic environments. Traditional robots are limited by fixed functional attributes and cannot efficiently reconfigure their strategies in response to task failures or environmental changes. To overcome this limitation, we leverage large language models (LLMs) to generate and extend BTs dynamically, combining the reasoning and generalization power of LLMs with the modularity and recovery capability of BTs. The proposed framework consists of four interconnected modules task initialization, task assignment, BT update, and failure node detection which operate in a closed loop. Robots tick their BTs during execution, and upon encountering a failure node, they can either extend the tree locally or invoke a centralized virtual coordinator (Alex) to reassign subtasks and synchronize BTs across peers. This design enables long-term cooperative execution in heterogeneous teams. We validate the framework on 60 tasks across three simulated scenarios and in a real-world cafe environment with a robotic arm and a wheeled-legged robot. Results show that our method consistently outperforms baseline approaches in task success rate, robustness, and scalability, demonstrating its effectiveness for multi-robot collaboration in complex scenarios.

Figures (7)

• Figure 1: The framework consists of three main components: (i) Human instruction, where natural language commands are provided; (ii) Virtual robot (Alex), which decomposes tasks, allocates subtasks, and shares knowledge; and (iii) Heterogeneous robots, including drones, robotic arms, and quadrupeds, which execute their own behavior trees collaboratively.
• Figure 2: Overview of the proposed framework for automatic behavior tree (BT) construction and adaptation in heterogeneous multi-robot systems. The framework consists of four interconnected modules: (1) Task Initialization, where human instructions are translated into an initial BTs using an LLM and predefined node templates; (2) Task Assignment, where failure nodes trigger the central allocator (Alex) to reassign tasks based on robot capabilities; (3) Behavior Tree Update, where new subtrees are inserted or synchronized across robots to extend task execution strategies; and (4) Failure Node Detection, where robots continuously tick their BTs to monitor execution and identify bottlenecks. These modules operate in a closed-loop cycle, enabling long-term adaptability and robust collaboration in dynamic environments.
• Figure 3: Illustration of the prompt-based behavior tree construction pipeline. Human natural language instructions are first converted into condition/action node prompts, which are then assembled into a behavior tree for robotic execution.
• Figure 4: Process of failure node detection, storage, and reuse in behavior tree execution.
• Figure 5: Behavior tree extension and insertion process. When a failure node is encountered, the system generates an extended subtree through LLM-based reasoning. This subtree is then inserted into the existing behavior tree, enabling the robot to recover from failures and continue task execution.