Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

BTGenBot-2: Efficient Behavior Tree Generation with Small Language Models

Riccardo Andrea Izzo, Gianluca Bardaro, Matteo Matteucci

TL;DR

BTGenBot-2 introduces a 1B parameter open-source small language model that directly converts natural language task descriptions and robot action primitives into executable ROS2 compatible behavior trees encoded in XML. It leverages a compute-efficient QLoRA fine-tuning workflow on a synthetic 5204 BT dataset and implements inference-time and runtime validators to ensure robustness on resource-constrained robots. A first standardized 52-task BTBenchmark evaluated in NVIDIA Isaac Sim demonstrates that BTGenBot-2 outperforms larger models and the previous BTGenBot across functional and non-functional metrics, with up to 16x faster inference and strong zero-shot and one-shot performance. Real-robot validation on navigation and manipulation tasks confirms practical deployment viability, highlighting the approach as a scalable and reproducible solution for on-device LLM-driven BT generation in ROS2 ecosystems.

Abstract

Recent advances in robot learning increasingly rely on LLM-based task planning, leveraging their ability to bridge natural language with executable actions. While prior works showcased great performances, the widespread adoption of these models in robotics has been challenging as 1) existing methods are often closed-source or computationally intensive, neglecting the actual deployment on real-world physical systems, and 2) there is no universally accepted, plug-and-play representation for robotic task generation. Addressing these challenges, we propose BTGenBot-2, a 1B-parameter open-source small language model that directly converts natural language task descriptions and a list of robot action primitives into executable behavior trees in XML. Unlike prior approaches, BTGenBot-2 enables zero-shot BT generation, error recovery at inference and runtime, while remaining lightweight enough for resource-constrained robots. We further introduce the first standardized benchmark for LLM-based BT generation, covering 52 navigation and manipulation tasks in NVIDIA Isaac Sim. Extensive evaluations demonstrate that BTGenBot-2 consistently outperforms GPT-5, Claude Opus 4.1, and larger open-source models across both functional and non-functional metrics, achieving average success rates of 90.38% in zero-shot and 98.07% in one-shot, while delivering up to 16x faster inference compared to the previous BTGenBot.

BTGenBot-2: Efficient Behavior Tree Generation with Small Language Models

TL;DR

BTGenBot-2 introduces a 1B parameter open-source small language model that directly converts natural language task descriptions and robot action primitives into executable ROS2 compatible behavior trees encoded in XML. It leverages a compute-efficient QLoRA fine-tuning workflow on a synthetic 5204 BT dataset and implements inference-time and runtime validators to ensure robustness on resource-constrained robots. A first standardized 52-task BTBenchmark evaluated in NVIDIA Isaac Sim demonstrates that BTGenBot-2 outperforms larger models and the previous BTGenBot across functional and non-functional metrics, with up to 16x faster inference and strong zero-shot and one-shot performance. Real-robot validation on navigation and manipulation tasks confirms practical deployment viability, highlighting the approach as a scalable and reproducible solution for on-device LLM-driven BT generation in ROS2 ecosystems.

Abstract

Recent advances in robot learning increasingly rely on LLM-based task planning, leveraging their ability to bridge natural language with executable actions. While prior works showcased great performances, the widespread adoption of these models in robotics has been challenging as 1) existing methods are often closed-source or computationally intensive, neglecting the actual deployment on real-world physical systems, and 2) there is no universally accepted, plug-and-play representation for robotic task generation. Addressing these challenges, we propose BTGenBot-2, a 1B-parameter open-source small language model that directly converts natural language task descriptions and a list of robot action primitives into executable behavior trees in XML. Unlike prior approaches, BTGenBot-2 enables zero-shot BT generation, error recovery at inference and runtime, while remaining lightweight enough for resource-constrained robots. We further introduce the first standardized benchmark for LLM-based BT generation, covering 52 navigation and manipulation tasks in NVIDIA Isaac Sim. Extensive evaluations demonstrate that BTGenBot-2 consistently outperforms GPT-5, Claude Opus 4.1, and larger open-source models across both functional and non-functional metrics, achieving average success rates of 90.38% in zero-shot and 98.07% in one-shot, while delivering up to 16x faster inference compared to the previous BTGenBot.
Paper Structure (19 sections, 1 equation, 6 figures, 1 table)

This paper contains 19 sections, 1 equation, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Behavior Trees (BTs)
  4. LLMs in robotics
  5. BTs generation using LLMs
  6. Methodology
  7. Problem Formulation
  8. Dataset
  9. Fine-Tuning
  10. Error Handling and Recovery
  11. Inference Time Validator
  12. Runtime Validator
  13. Behavior Tree Benchmark
  14. Experiments and Results
  15. Data Curation Pipeline Effectiveness
  16. ...and 4 more sections

Figures (6)

  • Figure 1: We present BTGenBot-2, a 1B-parameter open-source small language model (SLM), trained on 5k natural language instructions and behavior tree pairs from a synthetic instruction-following dataset. BTGenBot-2 achieves state-of-the-art performance in generating directly executable behavior trees. We open-source our synthetic dataset, model weights, codebase, and the BT Benchmark to support dissemination and reproducibility.
  • Figure 2: Dataset generation. Starting with the TSE dataset, a new instruction-following dataset is created through four key steps: (1) cleanse the raw XML data, (2) for each original BT, use gpt‑4o‑mini to generate three variants, (3) repeat step 2 with the new dataset, (4) merge all resulting datasets while producing a natural-language description for each BT.
  • Figure 3: Dataset sample. A representative example from the generated instruction-following dataset with its three components: the instruction that provides system contextual information, the input comprising a natural-language task description and its corresponding robot actions, and the output showcasing the generated XML-based behavior tree.
  • Figure 4: Overview of the model architecture. The model takes as input a natural language task description of a robotic task along with the set of available robot action primitives, generating a ROS2-compatible BT in XML format. The model is adapted using a QLoRA adapter while keeping pre-trained parameters frozen. The generated BTs are initially validated at inference time, checking for syntax and action-space consistency before execution. Additionally, at runtime, an inline logger tracks stack traces and blackboard states, triggering subtree regeneration in case of errors.
  • Figure 5: Data scaling evaluation. Average ROUGE and BLEU scores (mean ± std) with increasing dataset size: ROUGE: 46.2 ± 1.94, 66.2 ± 1.94, 77.6 ± 0.80, 82.2 ± 0.75; BLEU: 27.4 ± 1.02, 48.6 ± 1.85, 68.0 ± 1.10, 74.8 ± 0.75, corresponding to 600, 1,413, 3,791, and 5,204 samples. Standard deviation evaluated across 5 runs with temperature=0.9
  • ...and 1 more figures