LLM-Based Behavior Tree Generation for Construction Machinery

TL;DR

The paper addresses automation of earthwork operations amid an aging workforce by leveraging large language models (LLMs) to generate Behavior Trees (BTs) for ROS2-TMS for Construction. It proposes a two-step workflow where an LLM first produces a high-level task plan with synchronization flags, then generates executable BTs using a template, with safety ensured by referencing known database parameters. The approach is validated in simulation across multiple LLMs and in real-world experiments with an excavator and a dump truck, demonstrating safe cooperative operation and correct sequencing. Results indicate the method can scale cooperative automation across heterogeneous construction machines and reduce the manual BT design burden, highlighting practical CPS integration for civil engineering automation.

Abstract

Earthwork operations are facing an increasing demand, while workforce aging and skill loss create a pressing need for automation. ROS2-TMS for Construction, a Cyber-Physical System framework designed to coordinate construction machinery, has been proposed for autonomous operation; however, its reliance on manually designed Behavior Trees (BTs) limits scalability, particularly in scenarios involving heterogeneous machine cooperation. Recent advances in large language models (LLMs) offer new opportunities for task planning and BT generation. However, most existing approaches remain confined to simulations or simple manipulators, with relatively few applications demonstrated in real-world contexts, such as complex construction sites involving multiple machines. This paper proposes an LLM-based workflow for BT generation, introducing synchronization flags to enable safe and cooperative operation. The workflow consists of two steps: high-level planning, where the LLM generates synchronization flags, and BT generation using structured templates. Safety is ensured by planning with parameters stored in the system database. The proposed method is validated in simulation and further demonstrated through real-world experiments, highlighting its potential to advance automation in civil engineering.

Figures (7)

• Figure 1: The architecture of ROS2-TMS for Construction ROS2TMSforConstructionKasahara2024
• Figure 2: Our proposed workflow for achieving coordinated operations between the excavator and the dump truck. The pink boxes indicate knowledge provided to the LLM as prompts, the green boxes represent data input to and output from the database, and the blue boxes denote content generated by the LLM. The flags are managed on the Global Blackboard.
• Figure 3: Example of high-level task decomposition for the instruction “Move the dump truck next to the excavator.” A new flag, ZX200_INITIAL_POSE_FLG, is introduced in this case. Here, ZX200 and MST110CR are the names of the construction machines used in this study, representing an excavator and a dump truck respectively. The argument load_point indicates the soil loading location, which corresponds to the position next to the excavator. The blue text following the # symbol is the reasoning for that line.
• Figure 4: Template structure used to generate sequential BTs. The DBReader node reads a specified flag from DB into the Blackboard. The ConditionalExpression node immediately returns SUCCESS if the specified conditional expression evaluates to True, and FAILURE if it evaluates to False. Both of these nodes are custom nodes developed for ROS2-TMS for Construction. The flow at the bottom illustrates the tick propagation within the template structure.
• Figure 5: Simulation results of Scenario 4 using the Behavior Tree generated by GPT-5. The dump truck approaches the excavator (1–3); the excavator digs soil and loads it onto the dump truck (4–8). After the excavator returns to its initial pose (9–10), the dump truck moves to the dumping position and dumps the soil (11–12).